[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:58:56.043859  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:39105
I20260709 12:58:56.045347  7638 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:58:56.046319  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:56.060788  7645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:56.061534  7644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:56.062840  7638 server_base.cc:1061] running on GCE node
W20260709 12:58:56.062322  7647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.256984  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:57.257223  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.257514  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601937257487 us; error 0 us; skew 500 ppm
I20260709 12:58:57.258390  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.264022  7638 webserver.cc:533] Webserver started at http://127.7.117.190:34425/ using document root <none> and password file <none>
I20260709 12:58:57.264971  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.265215  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.265769  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.271544  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "ebe955e7d86445fd89a7eb040a90b6bc"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.279278  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:58:57.284391  7653 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.285532  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:58:57.285872  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "ebe955e7d86445fd89a7eb040a90b6bc"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.286192  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.347397  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.348995  7638 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:58:57.349478  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.421598  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:39105
I20260709 12:58:57.421674  7704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:39105 every 8 connection(s)
I20260709 12:58:57.429199  7705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:57.449972  7705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: Bootstrap starting.
I20260709 12:58:57.455734  7705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:57.457577  7705 log.cc:826] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: Log is configured to *not* fsync() on all Append() calls
I20260709 12:58:57.461956  7705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: No bootstrap required, opened a new log
I20260709 12:58:57.482107  7705 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER }
I20260709 12:58:57.482656  7705 raft_consensus.cc:385] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:57.482918  7705 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe955e7d86445fd89a7eb040a90b6bc, State: Initialized, Role: FOLLOWER
I20260709 12:58:57.483873  7705 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER }
I20260709 12:58:57.484355  7705 raft_consensus.cc:399] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:58:57.484620  7705 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:58:57.484997  7705 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:57.488997  7705 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER }
I20260709 12:58:57.489732  7705 leader_election.cc:304] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe955e7d86445fd89a7eb040a90b6bc; no voters: 
I20260709 12:58:57.491479  7705 leader_election.cc:290] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:58:57.491919  7708 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:57.494134  7708 raft_consensus.cc:697] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 1 LEADER]: Becoming Leader. State: Replica: ebe955e7d86445fd89a7eb040a90b6bc, State: Running, Role: LEADER
I20260709 12:58:57.495123  7708 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER }
I20260709 12:58:57.496338  7705 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:58:57.506945  7709 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER } }
I20260709 12:58:57.507277  7710 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe955e7d86445fd89a7eb040a90b6bc. Latest consensus state: current_term: 1 leader_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe955e7d86445fd89a7eb040a90b6bc" member_type: VOTER } }
I20260709 12:58:57.507745  7709 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:57.508055  7710 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:57.518880  7717 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:58:57.520782  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:58:57.527915  7717 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:58:57.543491  7717 catalog_manager.cc:1383] Generated new cluster ID: bf3263d5948447538e2ccf38c5c65de7
I20260709 12:58:57.543776  7717 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:58:57.571772  7717 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:58:57.573278  7717 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:58:57.585585  7717 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: Generated new TSK 0
I20260709 12:58:57.586630  7717 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:58:57.652812  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:57.667403  7728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.668161  7727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.673904  7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.674197  7638 server_base.cc:1061] running on GCE node
I20260709 12:58:57.675520  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:57.675900  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.676164  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601937676139 us; error 0 us; skew 500 ppm
I20260709 12:58:57.677171  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.682397  7638 webserver.cc:533] Webserver started at http://127.7.117.129:42085/ using document root <none> and password file <none>
I20260709 12:58:57.683116  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.683373  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.683894  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.685482  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "8ae79f6371ea42838a71e36f48d6361c"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.692941  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:58:57.698549  7735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.699481  7638 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:58:57.699878  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "8ae79f6371ea42838a71e36f48d6361c"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.700320  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.735304  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.736619  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.739483  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:57.744148  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:57.744403  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.744729  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:57.744884  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.849273  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:38627
I20260709 12:58:57.849381  7797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:38627 every 8 connection(s)
I20260709 12:58:57.856490  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:57.871556  7802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.873363  7803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.891358  7798 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39105
I20260709 12:58:57.891911  7798 heartbeater.cc:461] Registering TS with master...
W20260709 12:58:57.892077  7805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.892629  7638 server_base.cc:1061] running on GCE node
I20260709 12:58:57.893167  7798 heartbeater.cc:507] Master 127.7.117.190:39105 requested a full tablet report, sending...
I20260709 12:58:57.893638  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:57.893867  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.894058  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601937894042 us; error 0 us; skew 500 ppm
I20260709 12:58:57.894809  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.896347  7670 ts_manager.cc:194] Registered new tserver with Master: 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627)
I20260709 12:58:57.899123  7670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41916
I20260709 12:58:57.899860  7638 webserver.cc:533] Webserver started at http://127.7.117.130:38493/ using document root <none> and password file <none>
I20260709 12:58:57.900544  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.900838  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.901181  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.902484  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "35006a911d014dd7914d8a9e6bf23548"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.908103  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:58:57.912854  7810 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.913733  7638 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:58:57.914060  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "35006a911d014dd7914d8a9e6bf23548"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-ndzg"
I20260709 12:58:57.914436  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.939096  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.940757  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.943265  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:57.946838  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:57.947098  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.947379  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:57.947574  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:58.032049  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:39009
I20260709 12:58:58.032146  7872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:39009 every 8 connection(s)
I20260709 12:58:58.039044  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:58.051121  7876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:58.051594  7877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:58.052578  7873 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39105
I20260709 12:58:58.053154  7873 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:58.054234  7873 heartbeater.cc:507] Master 127.7.117.190:39105 requested a full tablet report, sending...
I20260709 12:58:58.057169  7670 ts_manager.cc:194] Registered new tserver with Master: 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:58.059026  7670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41926
I20260709 12:58:58.060810  7638 server_base.cc:1061] running on GCE node
W20260709 12:58:58.060782  7879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:58.062044  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:58.062309  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:58.062510  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601938062492 us; error 0 us; skew 500 ppm
I20260709 12:58:58.063186  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:58.065811  7638 webserver.cc:533] Webserver started at http://127.7.117.131:41945/ using document root <none> and password file <none>
I20260709 12:58:58.066407  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:58.066636  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:58.066937  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:58.068279  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "c4c59fca9045490da70c5bf702d8570f"
format_stamp: "Formatted at 2026-07-09 12:58:58 on dist-test-slave-ndzg"
I20260709 12:58:58.073870  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 12:58:58.077957  7884 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:58.078781  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 12:58:58.079097  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "c4c59fca9045490da70c5bf702d8570f"
format_stamp: "Formatted at 2026-07-09 12:58:58 on dist-test-slave-ndzg"
I20260709 12:58:58.079443  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:58.111658  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:58.113023  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:58.114419  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:58.116506  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:58.116763  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:58.117028  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:58.117201  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:58.214375  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:35705
I20260709 12:58:58.214464  7946 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:35705 every 8 connection(s)
I20260709 12:58:58.231752  7947 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39105
I20260709 12:58:58.232117  7947 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:58.232883  7947 heartbeater.cc:507] Master 127.7.117.190:39105 requested a full tablet report, sending...
I20260709 12:58:58.234782  7670 ts_manager.cc:194] Registered new tserver with Master: c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:58.234923  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015323702s
I20260709 12:58:58.236105  7670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41930
I20260709 12:58:58.287977  7670 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41940:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:58:58.291499  7670 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:58:58.407387  7838 tablet_service.cc:1511] Processing CreateTablet for tablet a849d52cfe52460b94d3c899f87872c2 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:58.407410  7837 tablet_service.cc:1511] Processing CreateTablet for tablet 73324d0f4c254c67b7c3b4c1513e44c8 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:58.407822  7835 tablet_service.cc:1511] Processing CreateTablet for tablet 285adfb485844514ab2770ac4bf99504 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:58.409677  7834 tablet_service.cc:1511] Processing CreateTablet for tablet a5c0373563704d8897e4d4e971868ca1 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:58.409904  7836 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcedcca97324038a98ef7187dc26329 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:58.411479  7833 tablet_service.cc:1511] Processing CreateTablet for tablet d273d63d3af14bb3a203cc435e33b2e5 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.412014  7831 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3b4b44ae41499f86332f9c3bf2110a (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.413944  7830 tablet_service.cc:1511] Processing CreateTablet for tablet bb843ba3301d437489597c804cbbbc07 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.415771  7832 tablet_service.cc:1511] Processing CreateTablet for tablet 342e0ba7a1774d158fa26f7503e24067 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.417853  7834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5c0373563704d8897e4d4e971868ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.418433  7830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb843ba3301d437489597c804cbbbc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.419750  7836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcedcca97324038a98ef7187dc26329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.419677  7762 tablet_service.cc:1511] Processing CreateTablet for tablet 73324d0f4c254c67b7c3b4c1513e44c8 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:58.421069  7762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73324d0f4c254c67b7c3b4c1513e44c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.421856  7763 tablet_service.cc:1511] Processing CreateTablet for tablet a849d52cfe52460b94d3c899f87872c2 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:58.423210  7763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a849d52cfe52460b94d3c899f87872c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.424233  7759 tablet_service.cc:1511] Processing CreateTablet for tablet a5c0373563704d8897e4d4e971868ca1 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:58.425549  7759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5c0373563704d8897e4d4e971868ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.425885  7832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342e0ba7a1774d158fa26f7503e24067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.426586  7761 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcedcca97324038a98ef7187dc26329 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:58.427933  7761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcedcca97324038a98ef7187dc26329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.437675  7838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a849d52cfe52460b94d3c899f87872c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.435905  7756 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3b4b44ae41499f86332f9c3bf2110a (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.442113  7755 tablet_service.cc:1511] Processing CreateTablet for tablet bb843ba3301d437489597c804cbbbc07 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.438882  7758 tablet_service.cc:1511] Processing CreateTablet for tablet d273d63d3af14bb3a203cc435e33b2e5 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.445037  7760 tablet_service.cc:1511] Processing CreateTablet for tablet 285adfb485844514ab2770ac4bf99504 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:58.447038  7758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d273d63d3af14bb3a203cc435e33b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.445314  7757 tablet_service.cc:1511] Processing CreateTablet for tablet 342e0ba7a1774d158fa26f7503e24067 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.449422  7831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3b4b44ae41499f86332f9c3bf2110a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.451998  7760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285adfb485844514ab2770ac4bf99504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.453208  7757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342e0ba7a1774d158fa26f7503e24067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.455852  7755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb843ba3301d437489597c804cbbbc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.456859  7756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3b4b44ae41499f86332f9c3bf2110a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.459928  7912 tablet_service.cc:1511] Processing CreateTablet for tablet a849d52cfe52460b94d3c899f87872c2 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:58.460274  7911 tablet_service.cc:1511] Processing CreateTablet for tablet 73324d0f4c254c67b7c3b4c1513e44c8 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:58.456444  7835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285adfb485844514ab2770ac4bf99504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.461822  7912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a849d52cfe52460b94d3c899f87872c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.476552  7833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d273d63d3af14bb3a203cc435e33b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.481387  7911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73324d0f4c254c67b7c3b4c1513e44c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.487066  7906 tablet_service.cc:1511] Processing CreateTablet for tablet 342e0ba7a1774d158fa26f7503e24067 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.482899  7908 tablet_service.cc:1511] Processing CreateTablet for tablet a5c0373563704d8897e4d4e971868ca1 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:58.495651  7907 tablet_service.cc:1511] Processing CreateTablet for tablet d273d63d3af14bb3a203cc435e33b2e5 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.497833  7905 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3b4b44ae41499f86332f9c3bf2110a (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.493461  7910 tablet_service.cc:1511] Processing CreateTablet for tablet 8fcedcca97324038a98ef7187dc26329 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:58.500643  7904 tablet_service.cc:1511] Processing CreateTablet for tablet bb843ba3301d437489597c804cbbbc07 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.503101  7906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342e0ba7a1774d158fa26f7503e24067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.503050  7909 tablet_service.cc:1511] Processing CreateTablet for tablet 285adfb485844514ab2770ac4bf99504 (DEFAULT_TABLE table=test-workload [id=daf949db2f0146cbb5481b17777390b4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:58.501705  7837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73324d0f4c254c67b7c3b4c1513e44c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.510310  7904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb843ba3301d437489597c804cbbbc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.529824  7909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285adfb485844514ab2770ac4bf99504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.534462  7907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d273d63d3af14bb3a203cc435e33b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:58:58.540328  7718 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:58:58.558676  7910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fcedcca97324038a98ef7187dc26329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.571532  7965 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.573092  7905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3b4b44ae41499f86332f9c3bf2110a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.591182  7908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5c0373563704d8897e4d4e971868ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.625981  7966 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.640188  7965 tablet_bootstrap.cc:654] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.682476  7967 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:58.704936  7966 tablet_bootstrap.cc:654] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.712065  7965 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.712772  7965 ts_tablet_manager.cc:1403] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.192s	user 0.031s	sys 0.021s
I20260709 12:58:58.717239  7965 raft_consensus.cc:359] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.718048  7965 raft_consensus.cc:385] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.718340  7965 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.719153  7965 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.724170  7967 tablet_bootstrap.cc:654] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.730350  7873 heartbeater.cc:499] Master 127.7.117.190:39105 was elected leader, sending a full tablet report...
I20260709 12:58:58.730530  7965 ts_tablet_manager.cc:1434] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.017s	user 0.000s	sys 0.008s
I20260709 12:58:58.732662  7965 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.740877  7965 tablet_bootstrap.cc:654] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.751241  7966 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.751734  7966 ts_tablet_manager.cc:1403] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.141s	user 0.011s	sys 0.006s
I20260709 12:58:58.754750  7966 raft_consensus.cc:359] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.755523  7966 raft_consensus.cc:385] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.755874  7966 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.756639  7966 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.766172  7966 ts_tablet_manager.cc:1434] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:58:58.766947  7798 heartbeater.cc:499] Master 127.7.117.190:39105 was elected leader, sending a full tablet report...
I20260709 12:58:58.767135  7966 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.772122  7967 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:58.772841  7967 ts_tablet_manager.cc:1403] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.091s	user 0.020s	sys 0.023s
I20260709 12:58:58.774667  7966 tablet_bootstrap.cc:654] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.777017  7967 raft_consensus.cc:359] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.778613  7967 raft_consensus.cc:385] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.780928  7967 raft_consensus.cc:740] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.781769  7967 consensus_queue.cc:260] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.782581  7965 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.783049  7965 ts_tablet_manager.cc:1403] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.051s	user 0.026s	sys 0.019s
I20260709 12:58:58.786587  7965 raft_consensus.cc:359] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.787302  7965 raft_consensus.cc:385] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.787596  7965 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.788343  7967 ts_tablet_manager.cc:1434] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 12:58:58.788308  7965 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.790868  7947 heartbeater.cc:499] Master 127.7.117.190:39105 was elected leader, sending a full tablet report...
I20260709 12:58:58.791057  7965 ts_tablet_manager.cc:1434] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
W20260709 12:58:58.791394  7874 tablet.cc:2406] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:58:58.791989  7965 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.793813  7966 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.794252  7966 ts_tablet_manager.cc:1403] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.011s
I20260709 12:58:58.797087  7966 raft_consensus.cc:359] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.797785  7966 raft_consensus.cc:385] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.798144  7966 raft_consensus.cc:740] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.798748  7965 tablet_bootstrap.cc:654] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.798962  7966 consensus_queue.cc:260] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.801278  7966 ts_tablet_manager.cc:1434] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:58:58.802304  7966 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.808588  7966 tablet_bootstrap.cc:654] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.809403  7967 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:58.820441  7966 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.820937  7966 ts_tablet_manager.cc:1403] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 12:58:58.822291  7965 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.822844  7965 ts_tablet_manager.cc:1403] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.010s
I20260709 12:58:58.823688  7966 raft_consensus.cc:359] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.824412  7966 raft_consensus.cc:385] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.824702  7966 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.825423  7966 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.827829  7966 ts_tablet_manager.cc:1434] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:58:58.828723  7966 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.825989  7965 raft_consensus.cc:359] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.829071  7965 raft_consensus.cc:385] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.829402  7965 raft_consensus.cc:740] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.830101  7965 consensus_queue.cc:260] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.832484  7965 ts_tablet_manager.cc:1434] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:58:58.833460  7965 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.834975  7966 tablet_bootstrap.cc:654] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.841759  7967 tablet_bootstrap.cc:654] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.843072  7965 tablet_bootstrap.cc:654] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.846796  7966 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.847270  7966 ts_tablet_manager.cc:1403] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.005s
I20260709 12:58:58.849992  7966 raft_consensus.cc:359] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.850709  7966 raft_consensus.cc:385] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.851044  7966 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.851771  7966 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.853718  7967 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:58.854223  7967 ts_tablet_manager.cc:1403] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.002s
I20260709 12:58:58.854225  7966 ts_tablet_manager.cc:1434] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:58:58.855605  7966 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.861939  7966 tablet_bootstrap.cc:654] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.863541  7967 raft_consensus.cc:359] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.864331  7967 raft_consensus.cc:385] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.864693  7967 raft_consensus.cc:740] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.866127  7965 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.866567  7965 ts_tablet_manager.cc:1403] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.006s
I20260709 12:58:58.865522  7967 consensus_queue.cc:260] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.870558  7965 raft_consensus.cc:359] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.871265  7965 raft_consensus.cc:385] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.871546  7965 raft_consensus.cc:740] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.872289  7965 consensus_queue.cc:260] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.874742  7966 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.874775  7965 ts_tablet_manager.cc:1434] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:58:58.875299  7966 ts_tablet_manager.cc:1403] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 12:58:58.875818  7965 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.877159  7967 ts_tablet_manager.cc:1434] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:58:58.878345  7966 raft_consensus.cc:359] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.879038  7966 raft_consensus.cc:385] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.879349  7966 raft_consensus.cc:740] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.880090  7966 consensus_queue.cc:260] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.880529  7967 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:58.882731  7966 ts_tablet_manager.cc:1434] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:58:58.883664  7966 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.885116  7965 tablet_bootstrap.cc:654] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.890057  7966 tablet_bootstrap.cc:654] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.890940  7967 tablet_bootstrap.cc:654] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.898435  7973 raft_consensus.cc:493] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.898957  7973 raft_consensus.cc:515] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.901693  7967 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:58.902170  7967 ts_tablet_manager.cc:1403] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 12:58:58.905687  7973 leader_election.cc:290] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:58.905789  7967 raft_consensus.cc:359] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.906757  7967 raft_consensus.cc:385] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.907096  7967 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.907955  7967 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.910249  7967 ts_tablet_manager.cc:1434] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:58:58.911187  7967 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:58.913272  7966 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:58.913759  7966 ts_tablet_manager.cc:1403] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.003s
I20260709 12:58:58.914763  7965 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.916635  7965 ts_tablet_manager.cc:1403] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.015s
I20260709 12:58:58.918673  7967 tablet_bootstrap.cc:654] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.921350  7965 raft_consensus.cc:359] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.922693  7973 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.922922  7965 raft_consensus.cc:385] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.923172  7973 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.923532  7965 raft_consensus.cc:740] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.925220  7973 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:58.924696  7965 consensus_queue.cc:260] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.928833  7965 ts_tablet_manager.cc:1434] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.006s
I20260709 12:58:58.929747  7966 raft_consensus.cc:359] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:58.930145  7965 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:58.930470  7966 raft_consensus.cc:385] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.930824  7966 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.939782  7965 tablet_bootstrap.cc:654] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.931557  7966 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:58.947093  7966 ts_tablet_manager.cc:1434] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:58:58.947996  7966 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:58.951961  7965 tablet_bootstrap.cc:492] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.952361  7973 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.952490  7965 ts_tablet_manager.cc:1403] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 12:58:58.952807  7973 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:58.954794  7973 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:58.956012  7965 raft_consensus.cc:359] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.956799  7965 raft_consensus.cc:385] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.957151  7965 raft_consensus.cc:740] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.958077  7965 consensus_queue.cc:260] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.959357  7847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:58.959721  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:58.960371  7848 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 0.
I20260709 12:58:58.961073  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
W20260709 12:58:58.962044  7737 leader_election.cc:343] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.962788  7737 leader_election.cc:304] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:58.962946  7965 ts_tablet_manager.cc:1434] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:58:58.964205  7965 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
W20260709 12:58:58.964787  7737 leader_election.cc:343] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.965418  7973 raft_consensus.cc:2804] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.965755  7973 raft_consensus.cc:493] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.966051  7973 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.971362  7965 tablet_bootstrap.cc:654] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.980306  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:58.980772  7920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:58.982329  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
W20260709 12:58:58.982775  7736 leader_election.cc:343] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.983246  7921 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 0.
W20260709 12:58:58.983729  7736 leader_election.cc:343] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.984125  7736 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c; no voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:58.988644  7965 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:58.977104  7973 raft_consensus.cc:515] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.989349  7969 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.990000  7967 tablet_bootstrap.cc:492] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:58.990473  7967 ts_tablet_manager.cc:1403] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.079s	user 0.043s	sys 0.014s
I20260709 12:58:58.990275  7969 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.992871  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:58.993448  7846 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.994014  7967 raft_consensus.cc:359] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.994776  7967 raft_consensus.cc:385] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.995061  7967 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.995944  7969 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:58.995887  7967 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:58.998728  7967 ts_tablet_manager.cc:1434] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:58:58.999907  7967 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:59.000346  7846 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 1.
I20260709 12:58:58.989707  7965 ts_tablet_manager.cc:1403] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.002s
I20260709 12:58:59.003775  7973 leader_election.cc:290] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.005671  7990 raft_consensus.cc:2749] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.008850  7973 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.007478  7967 tablet_bootstrap.cc:654] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.007392  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.008172  7736 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
W20260709 12:58:59.010912  7736 leader_election.cc:343] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.013069  7990 raft_consensus.cc:2804] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.013622  7990 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.014107  7990 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.014633  7737 leader_election.cc:304] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.009312  7973 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.018100  7965 raft_consensus.cc:359] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.021628  7965 raft_consensus.cc:385] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.021946  7965 raft_consensus.cc:740] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.022639  7965 consensus_queue.cc:260] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.020335  7966 tablet_bootstrap.cc:654] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.024528  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.025180  7846 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 0.
I20260709 12:58:59.026616  7737 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.031093  7993 raft_consensus.cc:2804] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.031859  7973 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.032974  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
W20260709 12:58:59.034377  7736 leader_election.cc:343] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.035344  7965 ts_tablet_manager.cc:1434] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:58:59.038794  7967 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:59.042193  7967 ts_tablet_manager.cc:1403] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.011s
I20260709 12:58:59.044960  7993 raft_consensus.cc:697] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.046932  7993 consensus_queue.cc:237] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.051576  7990 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.039597  7973 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.053267  7973 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.053705  7973 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.041909  7965 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:59.048666  7967 raft_consensus.cc:359] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.055183  7967 raft_consensus.cc:385] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.055492  7967 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.056176  7967 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.059818  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.061769  7967 ts_tablet_manager.cc:1434] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:58:59.062719  7967 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:59.062546  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
W20260709 12:58:59.064193  7737 leader_election.cc:343] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.066809  7966 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:59.067368  7966 ts_tablet_manager.cc:1403] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.002s
I20260709 12:58:59.069839  7967 tablet_bootstrap.cc:654] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.070391  7965 tablet_bootstrap.cc:654] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.070911  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.071558  7921 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.077916  7920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.078564  7921 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 1.
I20260709 12:58:59.079388  7990 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.079821  7736 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.080229  7965 tablet_bootstrap.cc:492] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:59.080679  7965 ts_tablet_manager.cc:1403] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.004s
W20260709 12:58:59.081357  7811 leader_election.cc:343] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.082312  7966 raft_consensus.cc:359] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.083019  7990 raft_consensus.cc:2804] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.080397  7973 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.083639  7966 raft_consensus.cc:385] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.084424  7966 raft_consensus.cc:740] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.084563  7990 raft_consensus.cc:697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.084746  7773 raft_consensus.cc:2393] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35006a911d014dd7914d8a9e6bf23548 in current term 1: Already voted for candidate 8ae79f6371ea42838a71e36f48d6361c in this term.
I20260709 12:58:59.085860  7966 consensus_queue.cc:260] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.086925  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.087512  7846 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.088894  7966 ts_tablet_manager.cc:1434] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:58:59.089771  7973 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.090703  7966 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:59.090770  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.092235  7990 consensus_queue.cc:237] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.095281  7975 raft_consensus.cc:493] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:59.097381  7736 leader_election.cc:343] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.096040  7975 raft_consensus.cc:515] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.103623  7975 leader_election.cc:290] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.103384  7965 raft_consensus.cc:359] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.104229  7965 raft_consensus.cc:385] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.104540  7965 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.098589  7811 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548; no voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.115223  7965 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.121990  7967 tablet_bootstrap.cc:492] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:59.122640  7967 ts_tablet_manager.cc:1403] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.022s
I20260709 12:58:59.124501  7966 tablet_bootstrap.cc:654] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.118132  7846 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 1.
I20260709 12:58:59.127923  7668 catalog_manager.cc:5697] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.132296  7967 raft_consensus.cc:359] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.133275  7967 raft_consensus.cc:385] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.133694  7967 raft_consensus.cc:740] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.134596  7967 consensus_queue.cc:260] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.135914  7975 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.136528  7975 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.139312  7975 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.140240  7969 raft_consensus.cc:2749] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.144057  7737 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.145473  7993 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.146062  7993 raft_consensus.cc:697] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.147068  7993 consensus_queue.cc:237] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.156665  7969 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.158749  7969 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.163514  7969 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.166356  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.167188  7773 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 0.
I20260709 12:58:59.171517  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.172290  7773 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 0.
I20260709 12:58:59.174229  7885 leader_election.cc:304] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.176591  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.176744  7975 raft_consensus.cc:2804] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.177366  7975 raft_consensus.cc:493] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.177591  7921 raft_consensus.cc:2393] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35006a911d014dd7914d8a9e6bf23548 in current term 1: Already voted for candidate 8ae79f6371ea42838a71e36f48d6361c in this term.
I20260709 12:58:59.177799  7975 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.174749  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.180331  7811 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548; no voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.182339  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.183022  7846 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 0.
I20260709 12:58:59.184052  7967 ts_tablet_manager.cc:1434] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.009s
I20260709 12:58:59.184619  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.185060  7967 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:59.185617  8007 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:58:59.186985  7886 leader_election.cc:343] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.187991  7885 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.195109  7965 ts_tablet_manager.cc:1434] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.112s	user 0.015s	sys 0.020s
I20260709 12:58:59.195987  7965 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: Bootstrap starting.
I20260709 12:58:59.191284  7966 tablet_bootstrap.cc:492] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:59.197831  7966 ts_tablet_manager.cc:1403] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.009s
I20260709 12:58:59.200390  7966 raft_consensus.cc:359] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.201049  7966 raft_consensus.cc:385] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.201324  7966 raft_consensus.cc:740] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.201978  7966 consensus_queue.cc:260] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.202518  7668 catalog_manager.cc:5697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.203902  7966 ts_tablet_manager.cc:1434] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:58:59.204752  7966 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: Bootstrap starting.
I20260709 12:58:59.205130  8008 raft_consensus.cc:2804] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.205603  8008 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.207100  8008 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.204753  7668 catalog_manager.cc:5697] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.209604  7965 tablet_bootstrap.cc:654] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.214557  7967 tablet_bootstrap.cc:654] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.213048  7975 raft_consensus.cc:515] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.216846  7993 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.217365  7993 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.219261  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.220664  7846 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.221452  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.221966  7773 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.219738  7975 leader_election.cc:290] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.224035  8008 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.227917  7966 tablet_bootstrap.cc:654] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.227917  8007 raft_consensus.cc:2749] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.228276  7993 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.231387  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.232010  7848 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 0.
I20260709 12:58:59.235586  8008 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.235707  7975 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.235641  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.236267  7975 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.236752  7772 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.236956  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.238607  7737 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.238727  7975 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
W20260709 12:58:59.238731  7886 leader_election.cc:343] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.240029  7993 raft_consensus.cc:2804] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.240427  7993 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.240784  7993 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.241333  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.241636  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.242730  7848 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 0.
I20260709 12:58:59.242854  7921 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 0.
I20260709 12:58:59.243870  7886 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.246315  7975 raft_consensus.cc:2804] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.246691  7975 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.247049  7975 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.246863  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.251857  7846 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 1.
I20260709 12:58:59.253026  8007 raft_consensus.cc:493] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.253141  7773 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 1.
I20260709 12:58:59.253310  7886 leader_election.cc:304] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.253511  8007 raft_consensus.cc:515] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.255721  8008 raft_consensus.cc:2804] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.261861  8007 leader_election.cc:290] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.264024  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.264647  7921 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 0.
I20260709 12:58:59.265684  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.265913  7811 leader_election.cc:304] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.266297  7773 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 0.
I20260709 12:58:59.267656  7993 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.270780  8007 raft_consensus.cc:2804] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.270481  7975 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.271549  8007 raft_consensus.cc:493] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.273808  8007 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.275368  7975 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.275882  7975 raft_consensus.cc:493] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.276453  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.276290  7975 raft_consensus.cc:515] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.276991  7846 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.273068  8004 raft_consensus.cc:493] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.277493  8004 raft_consensus.cc:515] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.278945  7771 raft_consensus.cc:2393] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4c59fca9045490da70c5bf702d8570f in current term 1: Already voted for candidate 8ae79f6371ea42838a71e36f48d6361c in this term.
I20260709 12:58:59.281571  7975 leader_election.cc:290] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.284014  7772 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 1.
I20260709 12:58:59.284677  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.285955  7965 tablet_bootstrap.cc:492] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: No bootstrap required, opened a new log
I20260709 12:58:59.286542  7965 ts_tablet_manager.cc:1403] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.020s
I20260709 12:58:59.284981  8004 leader_election.cc:290] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.285163  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342e0ba7a1774d158fa26f7503e24067" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.287506  7967 tablet_bootstrap.cc:492] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:59.288087  7967 ts_tablet_manager.cc:1403] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.003s
I20260709 12:58:59.290139  8008 raft_consensus.cc:697] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.290920  7771 raft_consensus.cc:2468] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 0.
I20260709 12:58:59.292652  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.293011  7993 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.295503  7847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.295854  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.296275  7772 raft_consensus.cc:2393] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c4c59fca9045490da70c5bf702d8570f in current term 1: Already voted for candidate 8ae79f6371ea42838a71e36f48d6361c in this term.
I20260709 12:58:59.298007  7920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.299556  7920 raft_consensus.cc:2393] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ae79f6371ea42838a71e36f48d6361c in current term 1: Already voted for candidate c4c59fca9045490da70c5bf702d8570f in this term.
I20260709 12:58:59.298710  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342e0ba7a1774d158fa26f7503e24067" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.300716  7846 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 1.
I20260709 12:58:59.301270  8007 raft_consensus.cc:515] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.302821  7966 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: No bootstrap required, opened a new log
I20260709 12:58:59.303184  7966 ts_tablet_manager.cc:1403] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.015s
I20260709 12:58:59.292897  8008 consensus_queue.cc:237] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.305209  7771 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 0.
I20260709 12:58:59.301918  7737 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c; no voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.303711  7848 raft_consensus.cc:2393] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4c59fca9045490da70c5bf702d8570f in current term 1: Already voted for candidate 35006a911d014dd7914d8a9e6bf23548 in this term.
I20260709 12:58:59.296283  7811 leader_election.cc:304] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.308050  7885 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
W20260709 12:58:59.308377  7811 leader_election.cc:343] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.308679  8004 raft_consensus.cc:2804] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.309049  8004 raft_consensus.cc:493] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.309471  8004 raft_consensus.cc:3060] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.310135  7886 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.306023  7965 raft_consensus.cc:359] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.305749  8017 raft_consensus.cc:493] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.311100  7965 raft_consensus.cc:385] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.301862  7967 raft_consensus.cc:359] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.310760  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.311394  7885 leader_election.cc:304] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.313112  7967 raft_consensus.cc:385] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.309460  8007 leader_election.cc:290] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.310967  7975 raft_consensus.cc:2804] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.313596  7967 raft_consensus.cc:740] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.311992  7965 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.312315  8016 raft_consensus.cc:2804] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.313235  7921 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.310230  7966 raft_consensus.cc:359] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.311861  7993 raft_consensus.cc:2749] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.314988  7966 raft_consensus.cc:385] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.315339  7966 raft_consensus.cc:740] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.313552  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.315883  7967 consensus_queue.cc:260] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.321837  7975 raft_consensus.cc:697] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.322662  7771 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.325063  7966 consensus_queue.cc:260] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.329608  7965 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.313566  8017 raft_consensus.cc:515] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.334118  7966 ts_tablet_manager.cc:1434] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.002s
I20260709 12:58:59.322898  7975 consensus_queue.cc:237] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.348119  8017 leader_election.cc:290] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.343837  8004 raft_consensus.cc:515] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.344159  7921 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 1.
I20260709 12:58:59.344377  7771 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 1.
I20260709 12:58:59.349546  7965 ts_tablet_manager.cc:1434] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: Time spent starting tablet: real 0.063s	user 0.001s	sys 0.012s
I20260709 12:58:59.351909  7811 leader_election.cc:304] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.354998  7967 ts_tablet_manager.cc:1434] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.007s
I20260709 12:58:59.355532  8004 leader_election.cc:290] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.356101  7967 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:59.356001  8007 raft_consensus.cc:2804] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.357515  8007 raft_consensus.cc:697] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:58:59.358501  8019 raft_consensus.cc:2764] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:58:59.358695  8007 consensus_queue.cc:237] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.357739  8016 raft_consensus.cc:697] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.362396  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342e0ba7a1774d158fa26f7503e24067" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.363446  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "342e0ba7a1774d158fa26f7503e24067" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.364497  7921 raft_consensus.cc:3060] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.364778  7772 raft_consensus.cc:3060] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.363160  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.368382  8016 consensus_queue.cc:237] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.380239  7772 raft_consensus.cc:2468] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 1.
I20260709 12:58:59.380539  7921 raft_consensus.cc:2468] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 1.
I20260709 12:58:59.384123  7811 leader_election.cc:304] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.385649  8004 raft_consensus.cc:493] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.385670  8007 raft_consensus.cc:2804] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:58:59.386379  7886 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:39009, user_credentials={real_user=slave}} blocked reactor thread for 74844.2us
I20260709 12:58:59.386252  8004 raft_consensus.cc:515] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.386564  8007 raft_consensus.cc:697] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:58:59.387705  7967 tablet_bootstrap.cc:654] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.387540  8007 consensus_queue.cc:237] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.388348  8004 leader_election.cc:290] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.390216  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.390974  7848 raft_consensus.cc:2393] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4c59fca9045490da70c5bf702d8570f in current term 1: Already voted for candidate 8ae79f6371ea42838a71e36f48d6361c in this term.
I20260709 12:58:59.392182  7886 leader_election.cc:304] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.392303  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.392923  8019 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.393877  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.394460  7772 raft_consensus.cc:2468] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 0.
I20260709 12:58:59.396188  7811 leader_election.cc:304] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.395813  7667 catalog_manager.cc:5697] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 1 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:59.397177  7811 leader_election.cc:343] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.397982  8007 raft_consensus.cc:2804] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.398342  8007 raft_consensus.cc:493] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.398679  8007 raft_consensus.cc:3060] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.399648  8019 raft_consensus.cc:2749] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.402653  7669 catalog_manager.cc:5697] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.405017  7669 catalog_manager.cc:5697] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.414997  8007 raft_consensus.cc:515] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.417709  8007 leader_election.cc:290] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.418668  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.420254  7967 tablet_bootstrap.cc:492] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:59.420677  7967 ts_tablet_manager.cc:1403] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.065s	user 0.007s	sys 0.007s
W20260709 12:58:59.421069  7811 leader_election.cc:343] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.423391  7967 raft_consensus.cc:359] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.423992  7967 raft_consensus.cc:385] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.424220  7967 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.424832  7967 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.426175  7667 catalog_manager.cc:5697] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 1 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.426733  7967 ts_tablet_manager.cc:1434] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:58:59.427680  7967 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: Bootstrap starting.
I20260709 12:58:59.427594  7667 catalog_manager.cc:5697] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 1 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:59.418270  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.429896  7772 raft_consensus.cc:3060] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.434012  7967 tablet_bootstrap.cc:654] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.436684  7772 raft_consensus.cc:2468] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 1.
I20260709 12:58:59.438095  7811 leader_election.cc:304] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.438941  8007 raft_consensus.cc:2804] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.439713  8007 raft_consensus.cc:697] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:58:59.440541  8007 consensus_queue.cc:237] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.441978  7967 tablet_bootstrap.cc:492] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: No bootstrap required, opened a new log
I20260709 12:58:59.442411  7967 ts_tablet_manager.cc:1403] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 12:58:59.444861  7967 raft_consensus.cc:359] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.445510  7967 raft_consensus.cc:385] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.445761  7967 raft_consensus.cc:740] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.446353  7967 consensus_queue.cc:260] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.448073  7967 ts_tablet_manager.cc:1434] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:58:59.454066  7669 catalog_manager.cc:5697] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.542470  7921 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.551440  7993 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.582141  7973 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.599390  7921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:35102
I20260709 12:58:59.600065  7921 raft_consensus.cc:606] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.610383  7921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:35102
I20260709 12:58:59.610973  7921 raft_consensus.cc:606] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.613252  7973 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.616921  7920 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "285adfb485844514ab2770ac4bf99504"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:35102
I20260709 12:58:59.617519  7920 raft_consensus.cc:606] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.630563  7993 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.635133  7772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:44954
I20260709 12:58:59.635788  7772 raft_consensus.cc:606] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.636981  7993 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.644127  7772 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:44954
I20260709 12:58:59.644682  7772 raft_consensus.cc:606] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.658110  7993 raft_consensus.cc:993] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:58:59.658555  7993 raft_consensus.cc:1081] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:58:59.664095  7990 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.673228  7846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50494
I20260709 12:58:59.673834  7846 raft_consensus.cc:493] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.674209  7846 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.675067  7847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:50502
I20260709 12:58:59.675621  7847 raft_consensus.cc:606] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Received request to transfer leadership to TS c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.681879  7846 raft_consensus.cc:515] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.683329  7848 raft_consensus.cc:1240] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Rejecting Update request from peer 8ae79f6371ea42838a71e36f48d6361c for earlier term 1. Current term is 2. Ops: []
I20260709 12:58:59.685431  7846 leader_election.cc:290] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 2 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.686497  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.686829  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.687165  7921 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.687436  7772 raft_consensus.cc:3055] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.687769  7772 raft_consensus.cc:740] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.688540  7772 consensus_queue.cc:260] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.689788  7772 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.693969  7921 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 2.
I20260709 12:58:59.695111  7811 leader_election.cc:304] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.696190  7772 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 2.
I20260709 12:58:59.696885  8007 raft_consensus.cc:2804] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.697350  8007 raft_consensus.cc:697] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:58:59.698472  8007 consensus_queue.cc:237] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.704605  7667 catalog_manager.cc:5697] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 1 to 2, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.743642  8007 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.744123  8007 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.745885  8007 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.746757  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.747051  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4c59fca9045490da70c5bf702d8570f" is_pre_election: true
I20260709 12:58:59.747395  7772 raft_consensus.cc:2393] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35006a911d014dd7914d8a9e6bf23548 in current term 1: Already voted for candidate c4c59fca9045490da70c5bf702d8570f in this term.
I20260709 12:58:59.748548  7811 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548; no voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f
I20260709 12:58:59.749182  8007 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.755604  8007 raft_consensus.cc:2749] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.758281  8019 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.770819  8021 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.780179  8016 raft_consensus.cc:993] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.780663  8021 raft_consensus.cc:1081] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.780633  8019 raft_consensus.cc:493] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.782181  7772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44950
I20260709 12:58:59.782742  7772 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.782541  8019 raft_consensus.cc:515] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.783113  7772 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.785794  8019 leader_election.cc:290] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.785948  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ae79f6371ea42838a71e36f48d6361c" is_pre_election: true
I20260709 12:58:59.786724  7771 raft_consensus.cc:2393] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4c59fca9045490da70c5bf702d8570f in current term 1: Already voted for candidate 35006a911d014dd7914d8a9e6bf23548 in this term.
I20260709 12:58:59.786826  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3b4b44ae41499f86332f9c3bf2110a" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35006a911d014dd7914d8a9e6bf23548" is_pre_election: true
I20260709 12:58:59.788380  7886 leader_election.cc:304] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:58:59.789532  8019 raft_consensus.cc:3060] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.790759  7772 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.792830  7772 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Requested vote from peers c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.794021  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.794293  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.794580  7921 raft_consensus.cc:3055] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.794878  7848 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.794951  7921 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.796777  7921 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.798197  7921 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.798591  8019 raft_consensus.cc:2749] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.801097  7848 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 2.
I20260709 12:58:59.802079  7737 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:58:59.802765  7973 raft_consensus.cc:2804] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.803226  7973 raft_consensus.cc:697] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.803879  7973 consensus_queue.cc:237] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:58:59.804625  7921 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 2.
I20260709 12:58:59.812099  8007 consensus_queue.cc:1048] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.814188  7669 catalog_manager.cc:5697] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 1 to 2, leader changed from c4c59fca9045490da70c5bf702d8570f (127.7.117.131) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.817752  8019 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.832294  8007 consensus_queue.cc:1048] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.833030  8004 consensus_queue.cc:1048] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.836732  8021 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.838785  8020 raft_consensus.cc:993] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.839221  8020 raft_consensus.cc:1081] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.843621  8004 consensus_queue.cc:1048] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.851733  7773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "285adfb485844514ab2770ac4bf99504"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44950
I20260709 12:58:59.852411  7773 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.852835  7773 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.861447  7773 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.864292  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.864843  7773 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.865020  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.865595  7921 raft_consensus.cc:3055] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.865927  7921 raft_consensus.cc:740] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.866811  7921 consensus_queue.cc:260] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.867913  7921 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.872498  8004 raft_consensus.cc:993] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:58:59.872997  8044 raft_consensus.cc:1081] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Signalling peer c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:58:59.875183  7920 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:35094
I20260709 12:58:59.875787  7920 raft_consensus.cc:493] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.876231  7920 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.883404  7921 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 2.
I20260709 12:58:59.888139  7736 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.889346  7973 raft_consensus.cc:2804] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.889851  7973 raft_consensus.cc:697] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.890720  7973 consensus_queue.cc:237] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.899498  8007 consensus_queue.cc:1048] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.900215  7668 catalog_manager.cc:5697] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 1 to 2, leader changed from c4c59fca9045490da70c5bf702d8570f (127.7.117.131) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.906136  7969 consensus_queue.cc:1048] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.884181  7920 raft_consensus.cc:515] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.912212  7920 leader_election.cc:290] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 2 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:58:59.914217  7771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:58:59.914719  7771 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.914834  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb843ba3301d437489597c804cbbbc07" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.915439  7848 raft_consensus.cc:3055] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.915771  7848 raft_consensus.cc:740] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:58:59.916523  7848 consensus_queue.cc:260] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.917757  7848 raft_consensus.cc:3060] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.921990  7771 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 2.
I20260709 12:58:59.923555  7885 leader_election.cc:304] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:58:59.924757  8019 raft_consensus.cc:2804] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.925417  8019 raft_consensus.cc:697] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.926862  7848 raft_consensus.cc:2468] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 2.
I20260709 12:58:59.926451  8019 consensus_queue.cc:237] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.932437  8020 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.946749  8019 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.947134  7667 catalog_manager.cc:5697] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 1 to 2, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 2 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:59.950222  8019 raft_consensus.cc:993] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.950729  8019 raft_consensus.cc:1081] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:58:59.956686  7772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44950
I20260709 12:58:59.957437  7772 raft_consensus.cc:493] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.957806  7772 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.966794  7772 raft_consensus.cc:515] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.970247  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:58:59.972957  7773 raft_consensus.cc:1240] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Rejecting Update request from peer c4c59fca9045490da70c5bf702d8570f for earlier term 1. Current term is 2. Ops: []
I20260709 12:58:59.974329  7772 leader_election.cc:290] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:58:59.974439  8019 consensus_queue.cc:1059] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.975772  8020 raft_consensus.cc:3055] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.976063  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:58:59.976094  8020 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:58:59.977138  8020 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.978837  8020 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.989511  7921 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 2.
I20260709 12:58:59.991184  7736 leader_election.cc:304] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 35006a911d014dd7914d8a9e6bf23548
I20260709 12:58:59.991855  7973 raft_consensus.cc:2804] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.992350  7973 raft_consensus.cc:697] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:58:59.993045  7973 consensus_queue.cc:237] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:58:59.999471  7667 catalog_manager.cc:5697] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 1 to 2, leader changed from c4c59fca9045490da70c5bf702d8570f (127.7.117.131) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.031613  7993 raft_consensus.cc:993] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:00.032162  7973 raft_consensus.cc:1081] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:00.033841  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50494
I20260709 12:59:00.034471  7848 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:00.034868  7848 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.039847  7848 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:00.042164  7848 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 2 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:00.042618  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:00.042734  7921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:00.043419  7773 raft_consensus.cc:3055] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:00.043433  7921 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.043939  7773 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:00.044723  7773 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:00.045738  7773 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.050225  7773 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 2.
I20260709 12:59:00.050246  7921 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 2.
I20260709 12:59:00.051371  7811 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, 8ae79f6371ea42838a71e36f48d6361c; no voters: 
I20260709 12:59:00.052090  8004 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:00.052560  8004 raft_consensus.cc:697] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:00.053367  8004 consensus_queue.cc:237] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:00.060158  7667 catalog_manager.cc:5697] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 1 to 2, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.245422  7848 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.246686  7973 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.256081  7920 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.256971  7921 raft_consensus.cc:1275] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.258674  8053 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.258737  8004 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.272442  7773 raft_consensus.cc:1275] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.273815  8004 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.342911  7848 raft_consensus.cc:1275] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.344221  8020 consensus_queue.cc:1048] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.351969  7773 raft_consensus.cc:1275] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.353565  8016 consensus_queue.cc:1048] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.404657  7920 raft_consensus.cc:1275] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.406011  8053 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.413189  7848 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.425336  7920 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.426919  7846 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.427219  7848 raft_consensus.cc:1275] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.426919  7990 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:00.428948  7990 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.438762  7846 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.440328  8053 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:00.458338  7773 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.459589  8007 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.468218  7920 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.471586  8007 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:02.738554  7638 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 12:59:02.739382  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:02.759675  7773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:54750
I20260709 12:59:02.760243  7773 raft_consensus.cc:606] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Received request to transfer leadership to TS c4c59fca9045490da70c5bf702d8570f
I20260709 12:59:02.771021  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:02.771531  7848 raft_consensus.cc:606] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Received request to transfer leadership to TS c4c59fca9045490da70c5bf702d8570f
I20260709 12:59:02.772704  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:02.773087  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:02.825106  8056 raft_consensus.cc:993] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:02.825626  8057 raft_consensus.cc:1081] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Signalling peer c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:02.826961  7922 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:35092
I20260709 12:59:02.827428  7922 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:02.827725  7922 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.832216  7922 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.833925  7922 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 2 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:59:02.834697  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:02.834990  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:02.835238  7773 raft_consensus.cc:3055] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:02.835502  7846 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.835574  7773 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:02.836457  7773 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.837410  7773 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.840142  7846 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 2.
I20260709 12:59:02.841243  7886 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:02.842012  8058 raft_consensus.cc:2804] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:02.842245  7773 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 2.
I20260709 12:59:02.842559  8058 raft_consensus.cc:697] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:02.843468  8058 consensus_queue.cc:237] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.855844  7669 catalog_manager.cc:5697] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 1 to 2, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 2 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:02.885492  8059 raft_consensus.cc:993] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:02.885969  8059 raft_consensus.cc:1081] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Signalling peer c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:02.887521  7922 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:35094
I20260709 12:59:02.887954  7922 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:02.888217  7922 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.893167  7922 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.894806  7922 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:59:02.895620  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:02.895866  7846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:02.896225  7773 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.896374  7846 raft_consensus.cc:3055] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:02.896699  7846 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:02.897372  7846 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.898274  7846 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.900907  7773 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 3.
I20260709 12:59:02.901986  7885 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:02.902673  8058 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:02.902918  7846 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 3.
I20260709 12:59:02.903172  8058 raft_consensus.cc:697] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:02.904021  8058 consensus_queue.cc:237] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:02.911660  7669 catalog_manager.cc:5697] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 2 to 3, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 3 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
W20260709 12:59:02.927757  7701 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 7641
W20260709 12:59:02.928550  7701 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 7704
W20260709 12:59:02.930408  7701 debug-util.cc:398] Leaking SignalData structure 0x7b08001ad120 after lost signal to thread 7946
I20260709 12:59:03.337982  7773 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:03.339433  8058 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:03.362519  7846 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:03.364171  8055 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:03.372316  7848 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:03.373870  8075 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:03.385046  7773 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:03.386725  8058 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:03.773881  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:03.785032  7773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:44954
I20260709 12:59:03.785635  7773 raft_consensus.cc:606] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:59:03.790971  7922 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:35102
I20260709 12:59:03.791472  7922 raft_consensus.cc:606] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:59:03.796893  7773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:44954
I20260709 12:59:03.797403  7773 raft_consensus.cc:606] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:59:03.802413  7773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "285adfb485844514ab2770ac4bf99504"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:44954
I20260709 12:59:03.802913  7773 raft_consensus.cc:606] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:59:03.808815  7922 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
mode: GRACEFUL
new_leader_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:35102
I20260709 12:59:03.809633  7922 raft_consensus.cc:606] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Received request to transfer leadership to TS 35006a911d014dd7914d8a9e6bf23548
I20260709 12:59:03.811203  8058 raft_consensus.cc:993] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.811617  8055 raft_consensus.cc:1081] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.813021  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50496
I20260709 12:59:03.813575  7848 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.813884  7848 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.823107  7848 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.825829  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:03.826121  7848 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 4 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:03.826786  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:03.827365  7922 raft_consensus.cc:3055] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:03.827594  7922 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:03.828125  7922 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.828933  7922 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.833460  7773 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.833392  7922 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 4.
I20260709 12:59:03.834818  7811 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:03.835712  8059 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:03.838689  7773 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 4.
I20260709 12:59:03.839937  8059 raft_consensus.cc:697] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:03.840916  8059 consensus_queue.cc:237] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.848599  7669 catalog_manager.cc:5697] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 3 to 4, leader changed from c4c59fca9045490da70c5bf702d8570f (127.7.117.131) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.904757  8084 raft_consensus.cc:993] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.905242  8056 raft_consensus.cc:1081] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.906941  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50494
I20260709 12:59:03.907418  7848 raft_consensus.cc:493] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.907714  7848 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.912552  7848 raft_consensus.cc:515] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.914263  7848 leader_election.cc:290] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:03.915030  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:03.915529  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:03.915625  7773 raft_consensus.cc:3055] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:03.916090  7773 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:03.916173  7922 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.916965  7773 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.918249  7773 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.920938  7922 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:03.922242  7811 leader_election.cc:304] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:03.922958  8059 raft_consensus.cc:2804] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:03.923409  8059 raft_consensus.cc:697] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:03.924216  8059 consensus_queue.cc:237] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.926414  7773 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:03.934556  8084 raft_consensus.cc:993] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.935052  8086 raft_consensus.cc:1081] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:03.935010  7669 catalog_manager.cc:5697] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 2 to 3, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.936857  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "285adfb485844514ab2770ac4bf99504"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50494
I20260709 12:59:03.937420  7848 raft_consensus.cc:493] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.937819  7848 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.944741  7848 raft_consensus.cc:515] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.946839  7848 leader_election.cc:290] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:03.947533  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:03.947793  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285adfb485844514ab2770ac4bf99504" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:03.948374  7773 raft_consensus.cc:3055] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:03.948432  7922 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.948864  7773 raft_consensus.cc:740] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:03.949663  7773 consensus_queue.cc:260] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.950855  7773 raft_consensus.cc:3060] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.955175  7922 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:03.956295  7811 leader_election.cc:304] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:03.957134  8080 raft_consensus.cc:2804] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:03.957374  7773 raft_consensus.cc:2468] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:03.957906  8080 raft_consensus.cc:697] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:03.958822  8080 consensus_queue.cc:237] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:03.967259  7669 catalog_manager.cc:5697] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 2 to 3, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:04.073735  8084 raft_consensus.cc:993] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:04.074223  8086 raft_consensus.cc:1081] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:04.075661  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50494
I20260709 12:59:04.076128  7848 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:04.076428  7848 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.080796  7848 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:04.082551  7848 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:04.083482  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:04.084074  7773 raft_consensus.cc:3055] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:04.083907  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:04.084394  7773 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:04.084563  7922 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.085216  7773 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:04.086476  7773 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.091390  7922 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:04.092583  7811 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:04.093117  7773 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:04.093488  8080 raft_consensus.cc:2804] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:04.094219  8080 raft_consensus.cc:697] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:04.095224  8080 consensus_queue.cc:237] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:04.102895  7669 catalog_manager.cc:5697] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 2 to 3, leader changed from 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:04.183315  8058 raft_consensus.cc:993] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: : Instructing follower 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:04.183827  8055 raft_consensus.cc:1081] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Signalling peer 35006a911d014dd7914d8a9e6bf23548 to start an election
I20260709 12:59:04.185439  7848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
 from {username='slave'} at 127.0.0.1:50496
I20260709 12:59:04.185881  7848 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:04.186158  7848 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.190169  7848 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:04.191704  7848 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:04.192487  7773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:04.192662  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "35006a911d014dd7914d8a9e6bf23548" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:04.193107  7773 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.193177  7922 raft_consensus.cc:3055] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:04.193629  7922 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:04.194327  7922 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:04.195161  7922 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:04.199900  7922 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:04.199847  7773 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35006a911d014dd7914d8a9e6bf23548 in term 3.
I20260709 12:59:04.201074  7811 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35006a911d014dd7914d8a9e6bf23548, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:04.201812  8080 raft_consensus.cc:2804] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:04.202308  8080 raft_consensus.cc:697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Leader. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:04.203142  8080 consensus_queue.cc:237] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:04.209561  7669 catalog_manager.cc:5697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 reported cstate change: term changed from 2 to 3, leader changed from c4c59fca9045490da70c5bf702d8570f (127.7.117.131) to 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "35006a911d014dd7914d8a9e6bf23548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:04.263700  7922 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:04.264946  8080 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:04.278980  7773 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:04.280666  8059 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:04.304942  7922 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.306248  8059 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.318372  7773 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.320104  8093 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.365845  7922 raft_consensus.cc:1275] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.367233  8093 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.378615  7773 raft_consensus.cc:1275] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.379978  8080 consensus_queue.cc:1048] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.496907  7922 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.498263  8096 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.509222  7773 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.510759  8059 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.590971  7773 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.592432  8059 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:04.603518  7922 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Refusing update from remote peer 35006a911d014dd7914d8a9e6bf23548: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.605046  8080 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:06.826768  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:06.834750  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:06.835469  7848 raft_consensus.cc:606] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:59:06.838042  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:06.838459  7848 raft_consensus.cc:606] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:59:06.840762  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:06.841167  7848 raft_consensus.cc:606] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 LEADER]: Received request to transfer leadership to TS c4c59fca9045490da70c5bf702d8570f
I20260709 12:59:06.843474  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:06.843957  7848 raft_consensus.cc:606] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Received request to transfer leadership to TS 8ae79f6371ea42838a71e36f48d6361c
I20260709 12:59:06.846298  7848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5"
dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
mode: GRACEFUL
new_leader_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:38112
I20260709 12:59:06.846745  7848 raft_consensus.cc:606] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Received request to transfer leadership to TS c4c59fca9045490da70c5bf702d8570f
I20260709 12:59:06.847752  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:06.848174  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:06.879995  8080 raft_consensus.cc:993] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:06.880432  8080 raft_consensus.cc:1081] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:06.881978  7772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44946
I20260709 12:59:06.882483  7772 raft_consensus.cc:493] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:06.882830  7772 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.887858  7772 raft_consensus.cc:515] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:06.889750  7772 leader_election.cc:290] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Requested vote from peers c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:59:06.890826  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:06.890719  7920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a849d52cfe52460b94d3c899f87872c2" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:06.891505  7848 raft_consensus.cc:3055] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:06.891579  7920 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.891866  7848 raft_consensus.cc:740] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:06.892771  7848 consensus_queue.cc:260] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:06.894026  7848 raft_consensus.cc:3060] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.898746  7920 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:06.900202  7736 leader_election.cc:304] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:06.900661  7848 raft_consensus.cc:2468] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:06.901968  8105 raft_consensus.cc:2804] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:06.902832  8105 raft_consensus.cc:697] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:06.903827  8105 consensus_queue.cc:237] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } }
I20260709 12:59:06.910530  7669 catalog_manager.cc:5697] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 3 to 4, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 4 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:06.931721  8104 raft_consensus.cc:993] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:06.932176  8104 raft_consensus.cc:1081] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Signalling peer c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:06.934017  7920 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:35094
I20260709 12:59:06.934444  7920 raft_consensus.cc:493] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:06.934769  7920 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:06.940059  7920 raft_consensus.cc:515] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:06.941654  7920 leader_election.cc:290] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 3 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:59:06.942431  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:06.942644  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d273d63d3af14bb3a203cc435e33b2e5" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:06.943030  7772 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:06.943150  7848 raft_consensus.cc:3055] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:06.943456  7848 raft_consensus.cc:740] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:06.944082  7848 consensus_queue.cc:260] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:06.944883  7848 raft_consensus.cc:3060] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:06.948225  7772 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 3.
I20260709 12:59:06.948840  7848 raft_consensus.cc:2468] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 3.
I20260709 12:59:06.949771  7885 leader_election.cc:304] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:06.950554  8055 raft_consensus.cc:2804] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:06.951628  8055 raft_consensus.cc:697] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:06.952346  8055 consensus_queue.cc:237] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:06.959908  7669 catalog_manager.cc:5697] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 2 to 3, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 3 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:07.046914  8080 raft_consensus.cc:993] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:07.047361  8104 raft_consensus.cc:1081] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:07.048822  7772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44946
I20260709 12:59:07.049304  7772 raft_consensus.cc:493] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:07.049615  7772 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.053997  7772 raft_consensus.cc:515] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.055500  7772 leader_election.cc:290] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:07.056332  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:07.056573  7920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fcedcca97324038a98ef7187dc26329" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:07.056861  7848 raft_consensus.cc:3055] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:07.057054  7920 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.057169  7848 raft_consensus.cc:740] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:07.057961  7848 consensus_queue.cc:260] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.059032  7848 raft_consensus.cc:3060] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.059800  8080 raft_consensus.cc:993] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:07.061138  8104 raft_consensus.cc:1081] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 LEADER]: Signalling peer c4c59fca9045490da70c5bf702d8570f to start an election
I20260709 12:59:07.063328  7922 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1"
dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
 from {username='slave'} at 127.0.0.1:35094
I20260709 12:59:07.064065  7922 raft_consensus.cc:493] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:07.063472  7920 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:07.064483  7922 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.065049  7848 raft_consensus.cc:2468] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:07.065241  7736 leader_election.cc:304] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:07.066401  8105 raft_consensus.cc:2804] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:07.066859  8105 raft_consensus.cc:697] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:07.067623  8105 consensus_queue.cc:237] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.069543  7922 raft_consensus.cc:515] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.071768  7922 leader_election.cc:290] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 5 election: Requested vote from peers 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627), 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009)
I20260709 12:59:07.072232  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:07.072825  7848 raft_consensus.cc:3055] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:59:07.072688  7772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5c0373563704d8897e4d4e971868ca1" candidate_uuid: "c4c59fca9045490da70c5bf702d8570f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
I20260709 12:59:07.073181  7848 raft_consensus.cc:740] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:07.073400  7772 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.073968  7848 consensus_queue.cc:260] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.075069  7848 raft_consensus.cc:3060] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.074700  7669 catalog_manager.cc:5697] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 3 to 4, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 4 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:07.079843  7772 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 5.
I20260709 12:59:07.081105  7848 raft_consensus.cc:2468] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c59fca9045490da70c5bf702d8570f in term 5.
I20260709 12:59:07.081120  7885 leader_election.cc:304] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:07.082167  8055 raft_consensus.cc:2804] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:59:07.082696  8055 raft_consensus.cc:697] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 5 LEADER]: Becoming Leader. State: Replica: c4c59fca9045490da70c5bf702d8570f, State: Running, Role: LEADER
I20260709 12:59:07.083503  8055 consensus_queue.cc:237] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.089766  7669 catalog_manager.cc:5697] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f reported cstate change: term changed from 4 to 5, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to c4c59fca9045490da70c5bf702d8570f (127.7.117.131). New cstate: current_term: 5 leader_uuid: "c4c59fca9045490da70c5bf702d8570f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:07.157403  8080 raft_consensus.cc:993] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: : Instructing follower 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:07.157932  8114 raft_consensus.cc:1081] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Signalling peer 8ae79f6371ea42838a71e36f48d6361c to start an election
I20260709 12:59:07.159302  7772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8"
dest_uuid: "8ae79f6371ea42838a71e36f48d6361c"
 from {username='slave'} at 127.0.0.1:44946
I20260709 12:59:07.159724  7772 raft_consensus.cc:493] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:07.159961  7772 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.163789  7772 raft_consensus.cc:515] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.165162  7772 leader_election.cc:290] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Requested vote from peers 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009), c4c59fca9045490da70c5bf702d8570f (127.7.117.131:35705)
I20260709 12:59:07.165923  7848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35006a911d014dd7914d8a9e6bf23548"
I20260709 12:59:07.166211  7922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73324d0f4c254c67b7c3b4c1513e44c8" candidate_uuid: "8ae79f6371ea42838a71e36f48d6361c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4c59fca9045490da70c5bf702d8570f"
I20260709 12:59:07.166397  7848 raft_consensus.cc:3055] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:07.166703  7848 raft_consensus.cc:740] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 35006a911d014dd7914d8a9e6bf23548, State: Running, Role: LEADER
I20260709 12:59:07.166712  7922 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.167443  7848 consensus_queue.cc:260] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.168315  7848 raft_consensus.cc:3060] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.170779  7922 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:07.171692  7736 leader_election.cc:304] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ae79f6371ea42838a71e36f48d6361c, c4c59fca9045490da70c5bf702d8570f; no voters: 
I20260709 12:59:07.172379  8105 raft_consensus.cc:2804] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:07.172495  7848 raft_consensus.cc:2468] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ae79f6371ea42838a71e36f48d6361c in term 4.
I20260709 12:59:07.172796  8105 raft_consensus.cc:697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Becoming Leader. State: Replica: 8ae79f6371ea42838a71e36f48d6361c, State: Running, Role: LEADER
I20260709 12:59:07.173692  8105 consensus_queue.cc:237] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } }
I20260709 12:59:07.180337  7669 catalog_manager.cc:5697] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c reported cstate change: term changed from 3 to 4, leader changed from 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130) to 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129). New cstate: current_term: 4 leader_uuid: "8ae79f6371ea42838a71e36f48d6361c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:07.313211  7922 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.314477  8105 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.324810  7848 raft_consensus.cc:1275] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.326578  8105 consensus_queue.cc:1048] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.375579  7772 raft_consensus.cc:1275] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:07.376820  8055 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:07.385864  7848 raft_consensus.cc:1275] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:07.387216  8124 consensus_queue.cc:1048] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:07.477967  7922 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.479115  8105 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.487955  7848 raft_consensus.cc:1275] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.489455  8127 consensus_queue.cc:1048] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:07.549988  7772 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 5 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:07.551280  8124 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ae79f6371ea42838a71e36f48d6361c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:07.559818  7848 raft_consensus.cc:1275] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 5 FOLLOWER]: Refusing update from remote peer c4c59fca9045490da70c5bf702d8570f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:07.561709  8112 consensus_queue.cc:1048] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:07.636369  7922 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.637566  8127 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c59fca9045490da70c5bf702d8570f" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.646212  7848 raft_consensus.cc:1275] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Refusing update from remote peer 8ae79f6371ea42838a71e36f48d6361c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.650278  8105 consensus_queue.cc:1048] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35006a911d014dd7914d8a9e6bf23548" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.849017  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:07.854357  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 12:59:07.885669  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:07.886410  7638 tablet_replica.cc:333] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.887146  7638 raft_consensus.cc:2243] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.887660  7638 raft_consensus.cc:2272] T 4d3b4b44ae41499f86332f9c3bf2110a P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.889922  7638 tablet_replica.cc:333] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.890427  7638 raft_consensus.cc:2243] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.890823  7638 raft_consensus.cc:2272] T 342e0ba7a1774d158fa26f7503e24067 P 8ae79f6371ea42838a71e36f48d6361c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.892613  7638 tablet_replica.cc:333] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.893270  7638 raft_consensus.cc:2243] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.894256  7638 raft_consensus.cc:2272] T a849d52cfe52460b94d3c899f87872c2 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:07.894608  7885 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:59:07.896929  7638 tablet_replica.cc:333] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.897580  7638 raft_consensus.cc:2243] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.898028  7638 raft_consensus.cc:2272] T bb843ba3301d437489597c804cbbbc07 P 8ae79f6371ea42838a71e36f48d6361c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:07.899209  7885 consensus_peers.cc:597] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f -> Peer 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627): Couldn't send request to peer 8ae79f6371ea42838a71e36f48d6361c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:07.899935  7638 tablet_replica.cc:333] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.900455  7638 raft_consensus.cc:2243] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.900982  7638 raft_consensus.cc:2272] T a5c0373563704d8897e4d4e971868ca1 P 8ae79f6371ea42838a71e36f48d6361c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.903626  7638 tablet_replica.cc:333] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.904145  7638 raft_consensus.cc:2243] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.904829  7638 raft_consensus.cc:2272] T 73324d0f4c254c67b7c3b4c1513e44c8 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.906852  7638 tablet_replica.cc:333] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.907356  7638 raft_consensus.cc:2243] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.907747  7638 raft_consensus.cc:2272] T 285adfb485844514ab2770ac4bf99504 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.909464  7638 tablet_replica.cc:333] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.909955  7638 raft_consensus.cc:2243] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.910364  7638 raft_consensus.cc:2272] T d273d63d3af14bb3a203cc435e33b2e5 P 8ae79f6371ea42838a71e36f48d6361c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.912467  7638 tablet_replica.cc:333] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c: stopping tablet replica
I20260709 12:59:07.913095  7638 raft_consensus.cc:2243] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.913846  7638 raft_consensus.cc:2272] T 8fcedcca97324038a98ef7187dc26329 P 8ae79f6371ea42838a71e36f48d6361c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.012401  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
W20260709 12:59:08.014703  7811 consensus_peers.cc:597] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 -> Peer 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627): Couldn't send request to peer 8ae79f6371ea42838a71e36f48d6361c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:08.033056  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
I20260709 12:59:08.056102  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:08.056923  7638 tablet_replica.cc:333] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.057940  7638 raft_consensus.cc:2243] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.058537  7638 raft_consensus.cc:2272] T 73324d0f4c254c67b7c3b4c1513e44c8 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.061398  7638 tablet_replica.cc:333] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.062209  7638 raft_consensus.cc:2243] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:59:08.062525  7886 consensus_peers.cc:597] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f -> Peer 35006a911d014dd7914d8a9e6bf23548 (127.7.117.130:39009): Couldn't send request to peer 35006a911d014dd7914d8a9e6bf23548. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:08.063146  7638 raft_consensus.cc:2272] T 4d3b4b44ae41499f86332f9c3bf2110a P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.065073  7638 tablet_replica.cc:333] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.065652  7638 raft_consensus.cc:2243] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.066443  7638 raft_consensus.cc:2272] T 285adfb485844514ab2770ac4bf99504 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.068073  7638 tablet_replica.cc:333] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.068583  7638 raft_consensus.cc:2243] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.068997  7638 raft_consensus.cc:2272] T 8fcedcca97324038a98ef7187dc26329 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.071192  7638 tablet_replica.cc:333] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.071738  7638 raft_consensus.cc:2243] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.072163  7638 raft_consensus.cc:2272] T d273d63d3af14bb3a203cc435e33b2e5 P 35006a911d014dd7914d8a9e6bf23548 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.074308  7638 tablet_replica.cc:333] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.074853  7638 raft_consensus.cc:2243] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.075264  7638 raft_consensus.cc:2272] T a5c0373563704d8897e4d4e971868ca1 P 35006a911d014dd7914d8a9e6bf23548 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.077241  7638 tablet_replica.cc:333] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.077831  7638 raft_consensus.cc:2243] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.078272  7638 raft_consensus.cc:2272] T bb843ba3301d437489597c804cbbbc07 P 35006a911d014dd7914d8a9e6bf23548 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.079932  7638 tablet_replica.cc:333] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.080446  7638 raft_consensus.cc:2243] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.080861  7638 raft_consensus.cc:2272] T a849d52cfe52460b94d3c899f87872c2 P 35006a911d014dd7914d8a9e6bf23548 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.082773  7638 tablet_replica.cc:333] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548: stopping tablet replica
I20260709 12:59:08.083237  7638 raft_consensus.cc:2243] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.083927  7638 raft_consensus.cc:2272] T 342e0ba7a1774d158fa26f7503e24067 P 35006a911d014dd7914d8a9e6bf23548 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:08.106150  7885 consensus_peers.cc:597] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f -> Peer 8ae79f6371ea42838a71e36f48d6361c (127.7.117.129:38627): Couldn't send request to peer 8ae79f6371ea42838a71e36f48d6361c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:08.109889  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
I20260709 12:59:08.126554  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
I20260709 12:59:08.147887  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:08.148509  7638 tablet_replica.cc:333] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.149103  7638 raft_consensus.cc:2243] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.149606  7638 raft_consensus.cc:2272] T 4d3b4b44ae41499f86332f9c3bf2110a P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.151312  7638 tablet_replica.cc:333] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.151783  7638 raft_consensus.cc:2243] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.152176  7638 raft_consensus.cc:2272] T 342e0ba7a1774d158fa26f7503e24067 P c4c59fca9045490da70c5bf702d8570f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.153781  7638 tablet_replica.cc:333] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.154263  7638 raft_consensus.cc:2243] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.154677  7638 raft_consensus.cc:2272] T a849d52cfe52460b94d3c899f87872c2 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.157186  7638 tablet_replica.cc:333] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.157681  7638 raft_consensus.cc:2243] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.158329  7638 raft_consensus.cc:2272] T d273d63d3af14bb3a203cc435e33b2e5 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.159894  7638 tablet_replica.cc:333] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.160339  7638 raft_consensus.cc:2243] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.160770  7638 raft_consensus.cc:2272] T 285adfb485844514ab2770ac4bf99504 P c4c59fca9045490da70c5bf702d8570f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.162348  7638 tablet_replica.cc:333] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.162817  7638 raft_consensus.cc:2243] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.163223  7638 raft_consensus.cc:2272] T 8fcedcca97324038a98ef7187dc26329 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.164963  7638 tablet_replica.cc:333] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.165469  7638 raft_consensus.cc:2243] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:08.165853  7638 raft_consensus.cc:2272] T 73324d0f4c254c67b7c3b4c1513e44c8 P c4c59fca9045490da70c5bf702d8570f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.167501  7638 tablet_replica.cc:333] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.167941  7638 raft_consensus.cc:2243] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.168613  7638 raft_consensus.cc:2272] T a5c0373563704d8897e4d4e971868ca1 P c4c59fca9045490da70c5bf702d8570f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.170323  7638 tablet_replica.cc:333] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f: stopping tablet replica
I20260709 12:59:08.170861  7638 raft_consensus.cc:2243] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.171633  7638 raft_consensus.cc:2272] T bb843ba3301d437489597c804cbbbc07 P c4c59fca9045490da70c5bf702d8570f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.197135  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 12:59:08.212690  7638 master.cc:562] Master@127.7.117.190:39105 shutting down...
I20260709 12:59:08.413333  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.413923  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.414284  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P ebe955e7d86445fd89a7eb040a90b6bc: stopping tablet replica
I20260709 12:59:08.451062  7638 master.cc:584] Master@127.7.117.190:39105 shutdown complete.
35006a911d014dd7914d8a9e6bf23548  4
8ae79f6371ea42838a71e36f48d6361c  4
c4c59fca9045490da70c5bf702d8570f  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12453 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:59:08.494390  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:46313
I20260709 12:59:08.495961  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.501876  8138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.502315  8137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.502545  8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.504570  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:08.505435  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.505610  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.505780  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601948505765 us; error 0 us; skew 500 ppm
I20260709 12:59:08.506315  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.508738  7638 webserver.cc:533] Webserver started at http://127.7.117.190:35043/ using document root <none> and password file <none>
I20260709 12:59:08.509220  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.509439  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.509754  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.510872  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "0f38dfa3f712443d8e544ed87ff54b20"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.515271  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 12:59:08.518919  8145 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.519764  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:59:08.520005  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "0f38dfa3f712443d8e544ed87ff54b20"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.520246  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.531127  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.532266  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.569244  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:46313
I20260709 12:59:08.569438  8196 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:46313 every 8 connection(s)
I20260709 12:59:08.573086  8197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.582168  8197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20: Bootstrap starting.
I20260709 12:59:08.586997  8197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:08.591090  8197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20: No bootstrap required, opened a new log
I20260709 12:59:08.593124  8197 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER }
I20260709 12:59:08.593586  8197 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:08.593811  8197 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f38dfa3f712443d8e544ed87ff54b20, State: Initialized, Role: FOLLOWER
I20260709 12:59:08.594426  8197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER }
I20260709 12:59:08.594884  8197 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:08.595100  8197 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:08.595374  8197 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:08.599402  8197 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER }
I20260709 12:59:08.599888  8197 leader_election.cc:304] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f38dfa3f712443d8e544ed87ff54b20; no voters: 
I20260709 12:59:08.601058  8197 leader_election.cc:290] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:08.601390  8200 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:08.602674  8200 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 1 LEADER]: Becoming Leader. State: Replica: 0f38dfa3f712443d8e544ed87ff54b20, State: Running, Role: LEADER
I20260709 12:59:08.603376  8200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER }
I20260709 12:59:08.603958  8197 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:08.608215  8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f38dfa3f712443d8e544ed87ff54b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER } }
I20260709 12:59:08.608992  8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:08.609592  8202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f38dfa3f712443d8e544ed87ff54b20. Latest consensus state: current_term: 1 leader_uuid: "0f38dfa3f712443d8e544ed87ff54b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f38dfa3f712443d8e544ed87ff54b20" member_type: VOTER } }
I20260709 12:59:08.609999  8202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:08.611261  8207 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:08.616048  8207 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:08.620281  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:08.624856  8207 catalog_manager.cc:1383] Generated new cluster ID: 3590e18afaec4419b934edb96d8a6b7d
I20260709 12:59:08.625155  8207 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:08.642596  8207 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:08.643903  8207 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:08.660949  8207 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20: Generated new TSK 0
I20260709 12:59:08.661576  8207 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:08.686630  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.692557  8218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.693357  8219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.697599  7638 server_base.cc:1061] running on GCE node
W20260709 12:59:08.697574  8221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.698732  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.699141  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.699342  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601948699322 us; error 0 us; skew 500 ppm
I20260709 12:59:08.699890  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.702286  7638 webserver.cc:533] Webserver started at http://127.7.117.129:34221/ using document root <none> and password file <none>
I20260709 12:59:08.702782  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.703002  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.703275  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.704309  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "4625def2990b484e9b6312f2721dcbd8"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.708588  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:59:08.711707  8226 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.712399  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:59:08.712718  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "4625def2990b484e9b6312f2721dcbd8"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.713060  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.727238  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.728447  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.730155  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.732354  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.732592  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.732826  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.733032  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.772110  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:39167
I20260709 12:59:08.772223  8288 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:39167 every 8 connection(s)
I20260709 12:59:08.777115  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.786614  8293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.786934  8294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.790591  8296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.791317  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:08.792441  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.792624  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.792822  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601948792807 us; error 0 us; skew 500 ppm
I20260709 12:59:08.793444  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.795732  7638 webserver.cc:533] Webserver started at http://127.7.117.130:43681/ using document root <none> and password file <none>
I20260709 12:59:08.796211  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.796414  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.796698  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.797892  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "b1a612823877416488065c9c19319105"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.799835  8289 heartbeater.cc:344] Connected to a master server at 127.7.117.190:46313
I20260709 12:59:08.800177  8289 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.800854  8289 heartbeater.cc:507] Master 127.7.117.190:46313 requested a full tablet report, sending...
I20260709 12:59:08.802839  8162 ts_manager.cc:194] Registered new tserver with Master: 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167)
I20260709 12:59:08.803117  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 12:59:08.804436  8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50648
I20260709 12:59:08.806880  8301 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.807739  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:59:08.808025  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "b1a612823877416488065c9c19319105"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.808291  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.821159  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.822670  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.824152  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.826375  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.826558  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.826846  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.826985  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.867394  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:35243
I20260709 12:59:08.867492  8363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:35243 every 8 connection(s)
I20260709 12:59:08.872056  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.879161  8367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.880672  8368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.882376  7638 server_base.cc:1061] running on GCE node
W20260709 12:59:08.883594  8370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.884676  8364 heartbeater.cc:344] Connected to a master server at 127.7.117.190:46313
I20260709 12:59:08.884676  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.885083  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.885120  8364 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.885324  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601948885279 us; error 0 us; skew 500 ppm
I20260709 12:59:08.885949  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.885996  8364 heartbeater.cc:507] Master 127.7.117.190:46313 requested a full tablet report, sending...
I20260709 12:59:08.888118  8162 ts_manager.cc:194] Registered new tserver with Master: b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:08.888460  7638 webserver.cc:533] Webserver started at http://127.7.117.131:34341/ using document root <none> and password file <none>
I20260709 12:59:08.888974  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.889189  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.889483  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.889598  8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50662
I20260709 12:59:08.890786  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "84e1de3cad67432bb32985503a774c2a"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.895066  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:59:08.898240  8375 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.899143  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:59:08.899528  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "84e1de3cad67432bb32985503a774c2a"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-ndzg"
I20260709 12:59:08.899850  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.919711  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.920918  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.922421  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.924510  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.924733  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.925043  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.925233  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.963828  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:43703
I20260709 12:59:08.963912  8437 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:43703 every 8 connection(s)
I20260709 12:59:08.976948  8438 heartbeater.cc:344] Connected to a master server at 127.7.117.190:46313
I20260709 12:59:08.977252  8438 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.977898  8438 heartbeater.cc:507] Master 127.7.117.190:46313 requested a full tablet report, sending...
I20260709 12:59:08.979645  8162 ts_manager.cc:194] Registered new tserver with Master: 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:08.980382  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013314226s
I20260709 12:59:08.980917  8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50678
I20260709 12:59:09.005239  8162 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50692:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:09.008215  8162 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:09.081424  8403 tablet_service.cc:1511] Processing CreateTablet for tablet d4b27a76e4da413595794b2b7b24fc56 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:09.083148  8403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b27a76e4da413595794b2b7b24fc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.087419  8402 tablet_service.cc:1511] Processing CreateTablet for tablet 5907404a3eb544b0882ee5210cf13709 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:09.087522  8401 tablet_service.cc:1511] Processing CreateTablet for tablet a88643dfe8014f518aa2b0163d2ce48e (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:09.088238  8400 tablet_service.cc:1511] Processing CreateTablet for tablet a46307364d724a5f90fe7b029b0746db (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:09.089054  8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88643dfe8014f518aa2b0163d2ce48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.092253  8398 tablet_service.cc:1511] Processing CreateTablet for tablet fcfa08578c0148ba866e24ff6dce61c1 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:09.093734  8399 tablet_service.cc:1511] Processing CreateTablet for tablet df9cbdc6d795433ab33c816e5e7552fc (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:09.095139  8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a46307364d724a5f90fe7b029b0746db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.095750  8402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5907404a3eb544b0882ee5210cf13709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.097724  8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9cbdc6d795433ab33c816e5e7552fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.101704  8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcfa08578c0148ba866e24ff6dce61c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.114367  8396 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7f285279264df0bd977ef1ecddef10 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:09.115787  8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7f285279264df0bd977ef1ecddef10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.117594  8327 tablet_service.cc:1511] Processing CreateTablet for tablet a88643dfe8014f518aa2b0163d2ce48e (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:09.117962  8254 tablet_service.cc:1511] Processing CreateTablet for tablet d4b27a76e4da413595794b2b7b24fc56 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:09.119222  8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b27a76e4da413595794b2b7b24fc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.119731  8394 tablet_service.cc:1511] Processing CreateTablet for tablet c98dc7c8003d499f95f18cbe168ce755 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:09.122151  8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98dc7c8003d499f95f18cbe168ce755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.122891  8252 tablet_service.cc:1511] Processing CreateTablet for tablet a88643dfe8014f518aa2b0163d2ce48e (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:09.124122  8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88643dfe8014f518aa2b0163d2ce48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.126726  8251 tablet_service.cc:1511] Processing CreateTablet for tablet a46307364d724a5f90fe7b029b0746db (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:09.128435  8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a46307364d724a5f90fe7b029b0746db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.130911  8323 tablet_service.cc:1511] Processing CreateTablet for tablet 6530acf16d6842098206f7555680e87d (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:09.131613  8328 tablet_service.cc:1511] Processing CreateTablet for tablet 5907404a3eb544b0882ee5210cf13709 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:09.133672  8247 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7f285279264df0bd977ef1ecddef10 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:09.134999  8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7f285279264df0bd977ef1ecddef10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.134871  8322 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7f285279264df0bd977ef1ecddef10 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:09.137676  8246 tablet_service.cc:1511] Processing CreateTablet for tablet e96a288a3eff4c3485e89b6f6f92507b (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:09.138465  8326 tablet_service.cc:1511] Processing CreateTablet for tablet a46307364d724a5f90fe7b029b0746db (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:09.139434  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96a288a3eff4c3485e89b6f6f92507b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.140376  8248 tablet_service.cc:1511] Processing CreateTablet for tablet 6530acf16d6842098206f7555680e87d (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:09.141376  8248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530acf16d6842098206f7555680e87d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.142474  8249 tablet_service.cc:1511] Processing CreateTablet for tablet fcfa08578c0148ba866e24ff6dce61c1 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:09.142884  8329 tablet_service.cc:1511] Processing CreateTablet for tablet d4b27a76e4da413595794b2b7b24fc56 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:09.143778  8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcfa08578c0148ba866e24ff6dce61c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.145459  8321 tablet_service.cc:1511] Processing CreateTablet for tablet e96a288a3eff4c3485e89b6f6f92507b (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:09.146317  8395 tablet_service.cc:1511] Processing CreateTablet for tablet e96a288a3eff4c3485e89b6f6f92507b (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:09.147643  8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96a288a3eff4c3485e89b6f6f92507b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.148082  8325 tablet_service.cc:1511] Processing CreateTablet for tablet df9cbdc6d795433ab33c816e5e7552fc (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:09.150825  8324 tablet_service.cc:1511] Processing CreateTablet for tablet fcfa08578c0148ba866e24ff6dce61c1 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:09.153512  8250 tablet_service.cc:1511] Processing CreateTablet for tablet df9cbdc6d795433ab33c816e5e7552fc (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:09.155284  8250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9cbdc6d795433ab33c816e5e7552fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.158838  8245 tablet_service.cc:1511] Processing CreateTablet for tablet c98dc7c8003d499f95f18cbe168ce755 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:09.160636  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98dc7c8003d499f95f18cbe168ce755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.162881  8320 tablet_service.cc:1511] Processing CreateTablet for tablet c98dc7c8003d499f95f18cbe168ce755 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:09.113044  8397 tablet_service.cc:1511] Processing CreateTablet for tablet 6530acf16d6842098206f7555680e87d (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:09.171092  8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530acf16d6842098206f7555680e87d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.172030  8327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88643dfe8014f518aa2b0163d2ce48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.169092  8253 tablet_service.cc:1511] Processing CreateTablet for tablet 5907404a3eb544b0882ee5210cf13709 (DEFAULT_TABLE table=test-workload [id=55fc25adce994752b7c118d08e8c8360]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:09.178797  8328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5907404a3eb544b0882ee5210cf13709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.180498  8325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9cbdc6d795433ab33c816e5e7552fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.182961  8329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b27a76e4da413595794b2b7b24fc56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.184350  8324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcfa08578c0148ba866e24ff6dce61c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.185906  8321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96a288a3eff4c3485e89b6f6f92507b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.186787  8320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98dc7c8003d499f95f18cbe168ce755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.188776  8323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530acf16d6842098206f7555680e87d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.190142  8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5907404a3eb544b0882ee5210cf13709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.193744  8326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a46307364d724a5f90fe7b029b0746db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.194855  8322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7f285279264df0bd977ef1ecddef10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:09.268796  8454 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.383291  8455 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.392392  8455 tablet_bootstrap.cc:654] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.418788  8454 tablet_bootstrap.cc:654] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.431089  8453 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.440378  8455 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.440907  8455 ts_tablet_manager.cc:1403] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.003s
I20260709 12:59:09.443702  8455 raft_consensus.cc:359] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.444393  8455 raft_consensus.cc:385] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.444694  8455 raft_consensus.cc:740] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.448501  8453 tablet_bootstrap.cc:654] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.450106  8455 consensus_queue.cc:260] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.453073  8364 heartbeater.cc:499] Master 127.7.117.190:46313 was elected leader, sending a full tablet report...
I20260709 12:59:09.453683  8454 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.454113  8454 ts_tablet_manager.cc:1403] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.010s
I20260709 12:59:09.456795  8454 raft_consensus.cc:359] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.457504  8454 raft_consensus.cc:385] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.457785  8454 raft_consensus.cc:740] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.459264  8455 ts_tablet_manager.cc:1434] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 12:59:09.460263  8455 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.461645  8454 consensus_queue.cc:260] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.466089  8453 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.466548  8453 ts_tablet_manager.cc:1403] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.004s
I20260709 12:59:09.466612  8455 tablet_bootstrap.cc:654] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.468883  8454 ts_tablet_manager.cc:1434] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.006s
I20260709 12:59:09.469624  8289 heartbeater.cc:499] Master 127.7.117.190:46313 was elected leader, sending a full tablet report...
I20260709 12:59:09.470188  8454 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.478003  8460 raft_consensus.cc:493] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.478631  8460 raft_consensus.cc:515] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.469614  8453 raft_consensus.cc:359] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.481523  8453 raft_consensus.cc:385] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.483757  8453 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.484979  8453 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.490796  8460 leader_election.cc:290] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.496450  8454 tablet_bootstrap.cc:654] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.505646  8455 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.506147  8455 ts_tablet_manager.cc:1403] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.020s
I20260709 12:59:09.508838  8455 raft_consensus.cc:359] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.509568  8455 raft_consensus.cc:385] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.509863  8455 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.510460  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.510572  8455 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.512095  8438 heartbeater.cc:499] Master 127.7.117.190:46313 was elected leader, sending a full tablet report...
I20260709 12:59:09.512885  8455 ts_tablet_manager.cc:1434] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:09.513896  8455 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: Bootstrap starting.
W20260709 12:59:09.514381  8228 leader_election.cc:343] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1a612823877416488065c9c19319105 (127.7.117.130:35243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.520267  8455 tablet_bootstrap.cc:654] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.523185  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:09.524750  8454 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
W20260709 12:59:09.524885  8230 leader_election.cc:343] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.525473  8454 ts_tablet_manager.cc:1403] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.056s	user 0.006s	sys 0.007s
I20260709 12:59:09.525626  8230 leader_election.cc:304] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8; no voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105
I20260709 12:59:09.526631  8462 raft_consensus.cc:2749] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:09.529242  8454 raft_consensus.cc:359] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.530194  8454 raft_consensus.cc:385] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.530566  8454 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.531190  8453 ts_tablet_manager.cc:1434] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.064s	user 0.019s	sys 0.011s
I20260709 12:59:09.531649  8454 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.532747  8453 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.535442  8455 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.535902  8455 ts_tablet_manager.cc:1403] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 12:59:09.539453  8454 ts_tablet_manager.cc:1434] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:59:09.539548  8453 tablet_bootstrap.cc:654] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.539510  8455 raft_consensus.cc:359] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.540369  8455 raft_consensus.cc:385] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.540644  8454 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.540675  8455 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.541532  8455 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.543851  8455 ts_tablet_manager.cc:1434] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:59:09.544857  8455 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.547417  8454 tablet_bootstrap.cc:654] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.552088  8455 tablet_bootstrap.cc:654] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.558413  8453 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.558936  8453 ts_tablet_manager.cc:1403] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.014s
I20260709 12:59:09.559855  8454 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.560443  8454 ts_tablet_manager.cc:1403] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:59:09.562009  8453 raft_consensus.cc:359] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.562718  8453 raft_consensus.cc:385] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.562975  8453 raft_consensus.cc:740] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.563338  8454 raft_consensus.cc:359] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.563846  8455 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.563616  8453 consensus_queue.cc:260] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.564062  8454 raft_consensus.cc:385] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.564296  8455 ts_tablet_manager.cc:1403] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.010s
I20260709 12:59:09.564404  8454 raft_consensus.cc:740] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.565663  8453 ts_tablet_manager.cc:1434] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 12:59:09.565340  8454 consensus_queue.cc:260] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.566535  8453 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.567198  8455 raft_consensus.cc:359] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.567898  8455 raft_consensus.cc:385] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.568298  8455 raft_consensus.cc:740] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.569162  8455 consensus_queue.cc:260] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.570122  8454 ts_tablet_manager.cc:1434] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:59:09.571094  8454 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.573249  8453 tablet_bootstrap.cc:654] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.573760  8462 raft_consensus.cc:493] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.574225  8455 ts_tablet_manager.cc:1434] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:59:09.574203  8462 raft_consensus.cc:515] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.575304  8455 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.576491  8462 leader_election.cc:290] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.577600  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:09.578162  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.578738  8339 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 0.
W20260709 12:59:09.578958  8230 leader_election.cc:343] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.582633  8455 tablet_bootstrap.cc:654] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.582644  8454 tablet_bootstrap.cc:654] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.584223  8228 leader_election.cc:304] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:09.585726  8462 raft_consensus.cc:2804] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.586102  8462 raft_consensus.cc:493] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.586437  8462 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.587743  8453 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.588203  8453 ts_tablet_manager.cc:1403] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.006s
I20260709 12:59:09.591049  8453 raft_consensus.cc:359] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.591740  8453 raft_consensus.cc:385] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.592003  8453 raft_consensus.cc:740] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.592891  8453 consensus_queue.cc:260] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.595661  8453 ts_tablet_manager.cc:1434] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:09.598906  8455 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.599457  8455 ts_tablet_manager.cc:1403] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.003s
I20260709 12:59:09.599972  8453 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.602252  8455 raft_consensus.cc:359] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.602942  8455 raft_consensus.cc:385] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.603201  8454 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.603238  8455 raft_consensus.cc:740] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.603753  8454 ts_tablet_manager.cc:1403] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.012s
I20260709 12:59:09.604180  8455 consensus_queue.cc:260] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.606555  8453 tablet_bootstrap.cc:654] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.608533  8454 raft_consensus.cc:359] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.609277  8454 raft_consensus.cc:385] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.609596  8454 raft_consensus.cc:740] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.610330  8454 consensus_queue.cc:260] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.599013  8462 raft_consensus.cc:515] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.613180  8455 ts_tablet_manager.cc:1434] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
W20260709 12:59:09.615170  8210 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:09.615710  8462 leader_election.cc:290] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.616382  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:09.616797  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:09.616945  8339 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:09.618078  8230 leader_election.cc:343] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.620064  8455 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.621249  8454 ts_tablet_manager.cc:1434] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:59:09.625252  8339 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 1.
I20260709 12:59:09.625795  8454 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.626518  8228 leader_election.cc:304] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:09.627393  8455 tablet_bootstrap.cc:654] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.627678  8464 raft_consensus.cc:493] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.628151  8464 raft_consensus.cc:515] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.636536  8462 raft_consensus.cc:2804] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.637014  8462 raft_consensus.cc:697] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:09.637684  8462 consensus_queue.cc:237] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.640892  8464 leader_election.cc:290] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:09.646808  8453 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.647437  8453 ts_tablet_manager.cc:1403] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.049s	user 0.006s	sys 0.010s
I20260709 12:59:09.652608  8454 tablet_bootstrap.cc:654] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.663527  8453 raft_consensus.cc:359] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.664536  8453 raft_consensus.cc:385] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.664903  8453 raft_consensus.cc:740] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.666225  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.666869  8339 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:09.668359  8377 leader_election.cc:304] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:09.667786  8162 catalog_manager.cc:5697] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.669243  8464 raft_consensus.cc:2804] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.669756  8464 raft_consensus.cc:493] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.670101  8464 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.671160  8462 raft_consensus.cc:493] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.671602  8462 raft_consensus.cc:515] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.672662  8453 consensus_queue.cc:260] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.673939  8455 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.674532  8455 ts_tablet_manager.cc:1403] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.036s
I20260709 12:59:09.675441  8462 leader_election.cc:290] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.677855  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:09.678531  8455 raft_consensus.cc:359] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
W20260709 12:59:09.679265  8230 leader_election.cc:343] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.679487  8455 raft_consensus.cc:385] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.679121  8464 raft_consensus.cc:515] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.679929  8455 raft_consensus.cc:740] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.680099  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.680722  8339 raft_consensus.cc:2468] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 0.
I20260709 12:59:09.680972  8455 consensus_queue.cc:260] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.681908  8228 leader_election.cc:304] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:09.683019  8462 raft_consensus.cc:2804] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.683406  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:09.683982  8339 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.686910  8455 ts_tablet_manager.cc:1434] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:59:09.688237  8455 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.683380  8462 raft_consensus.cc:493] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.689914  8462 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.693743  8464 leader_election.cc:290] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:09.696293  8454 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.696810  8454 ts_tablet_manager.cc:1403] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.003s
I20260709 12:59:09.697422  8339 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:09.698863  8455 tablet_bootstrap.cc:654] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.699190  8377 leader_election.cc:304] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:09.700103  8464 raft_consensus.cc:2804] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.699717  8454 raft_consensus.cc:359] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.700634  8454 raft_consensus.cc:385] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.694383  8453 ts_tablet_manager.cc:1434] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.047s	user 0.015s	sys 0.014s
I20260709 12:59:09.701339  8481 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.700977  8454 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.702108  8481 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.702791  8454 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.705987  8481 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:09.705919  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.706593  8454 ts_tablet_manager.cc:1434] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:59:09.707518  8454 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
W20260709 12:59:09.707470  8377 leader_election.cc:343] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1a612823877416488065c9c19319105 (127.7.117.130:35243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.700888  8464 raft_consensus.cc:697] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:09.709100  8464 consensus_queue.cc:237] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.705888  8453 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.715353  8455 tablet_bootstrap.cc:492] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.715808  8455 ts_tablet_manager.cc:1403] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.005s
I20260709 12:59:09.715629  8462 raft_consensus.cc:515] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.720822  8453 tablet_bootstrap.cc:654] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.720677  8455 raft_consensus.cc:359] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.721421  8455 raft_consensus.cc:385] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.721728  8455 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.722636  8455 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.725214  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:09.725940  8339 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.726362  8264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:09.727701  8455 ts_tablet_manager.cc:1434] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:09.727756  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:09.728761  8454 tablet_bootstrap.cc:654] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.728790  8455 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: Bootstrap starting.
W20260709 12:59:09.729126  8379 leader_election.cc:343] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.730351  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:09.731009  8262 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:09.732707  8379 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: b1a612823877416488065c9c19319105
W20260709 12:59:09.733958  8379 leader_election.cc:343] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.734875  8481 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.735217  8481 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.735561  8481 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.739262  8455 tablet_bootstrap.cc:654] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.742555  8481 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.743337  8339 raft_consensus.cc:2468] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 1.
I20260709 12:59:09.744536  8228 leader_election.cc:304] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:09.745397  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:09.745903  8453 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.746160  8263 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.746784  8453 ts_tablet_manager.cc:1403] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.012s
I20260709 12:59:09.745386  8460 raft_consensus.cc:2804] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.746889  8462 leader_election.cc:290] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.748361  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
W20260709 12:59:09.755486  8230 leader_election.cc:343] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.752993  8453 raft_consensus.cc:359] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.757174  8453 raft_consensus.cc:385] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.757687  8453 raft_consensus.cc:740] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.758919  8453 consensus_queue.cc:260] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.763744  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:09.764463  8339 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.765602  8455 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.766085  8455 ts_tablet_manager.cc:1403] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.011s
I20260709 12:59:09.769407  8455 raft_consensus.cc:359] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.770067  8455 raft_consensus.cc:385] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.770327  8455 raft_consensus.cc:740] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.771523  8453 ts_tablet_manager.cc:1434] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 12:59:09.771237  8455 consensus_queue.cc:260] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.772601  8453 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.773957  8481 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:09.776089  8263 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:09.776937  8455 ts_tablet_manager.cc:1434] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:09.777612  8379 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:09.779604  8453 tablet_bootstrap.cc:654] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.780318  8455 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.780310  8481 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.781138  8481 raft_consensus.cc:697] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:09.782812  8481 consensus_queue.cc:237] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.785228  8460 raft_consensus.cc:697] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:09.786394  8460 consensus_queue.cc:237] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.787904  8455 tablet_bootstrap.cc:654] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.792203  8454 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.792138  8339 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:09.792091  8453 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.792838  8454 ts_tablet_manager.cc:1403] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.086s	user 0.018s	sys 0.013s
I20260709 12:59:09.793318  8453 ts_tablet_manager.cc:1403] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 12:59:09.799311  8160 catalog_manager.cc:5697] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 1 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.806454  8454 raft_consensus.cc:359] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.807221  8454 raft_consensus.cc:385] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.807598  8454 raft_consensus.cc:740] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.807279  8453 raft_consensus.cc:359] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.808010  8453 raft_consensus.cc:385] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.808319  8453 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.808439  8454 consensus_queue.cc:260] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.809062  8453 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.812665  8454 ts_tablet_manager.cc:1434] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:59:09.813827  8454 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.815006  8453 ts_tablet_manager.cc:1434] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:59:09.816893  8453 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.820515  8454 tablet_bootstrap.cc:654] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.819489  8162 catalog_manager.cc:5697] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.822373  8453 tablet_bootstrap.cc:654] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.824680  8455 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.825191  8455 ts_tablet_manager.cc:1403] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.013s
I20260709 12:59:09.828872  8455 raft_consensus.cc:359] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.831660  8455 raft_consensus.cc:385] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.832095  8455 raft_consensus.cc:740] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.833201  8455 consensus_queue.cc:260] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.838404  8160 catalog_manager.cc:5697] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 1 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:09.841109  8455 ts_tablet_manager.cc:1434] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:59:09.848989  8455 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: Bootstrap starting.
I20260709 12:59:09.856401  8455 tablet_bootstrap.cc:654] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.880849  8454 tablet_bootstrap.cc:492] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.881400  8454 ts_tablet_manager.cc:1403] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.068s	user 0.037s	sys 0.027s
I20260709 12:59:09.882459  8455 tablet_bootstrap.cc:492] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: No bootstrap required, opened a new log
I20260709 12:59:09.882493  8453 tablet_bootstrap.cc:492] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.883029  8455 ts_tablet_manager.cc:1403] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.003s
I20260709 12:59:09.883033  8453 ts_tablet_manager.cc:1403] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.067s	user 0.021s	sys 0.012s
I20260709 12:59:09.883555  8454 raft_consensus.cc:359] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.884109  8454 raft_consensus.cc:385] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.884393  8454 raft_consensus.cc:740] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.885130  8454 consensus_queue.cc:260] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.885767  8455 raft_consensus.cc:359] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.886027  8453 raft_consensus.cc:359] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.886405  8455 raft_consensus.cc:385] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.886654  8453 raft_consensus.cc:385] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.886673  8455 raft_consensus.cc:740] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.887051  8453 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.887217  8454 ts_tablet_manager.cc:1434] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:09.887578  8455 consensus_queue.cc:260] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.888197  8454 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.887887  8453 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.890093  8453 ts_tablet_manager.cc:1434] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:09.891023  8453 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.894914  8454 tablet_bootstrap.cc:654] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.897146  8453 tablet_bootstrap.cc:654] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.897207  8455 ts_tablet_manager.cc:1434] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:59:09.903646  8453 tablet_bootstrap.cc:492] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.903947  8453 ts_tablet_manager.cc:1403] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 12:59:09.906136  8454 tablet_bootstrap.cc:492] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.906261  8453 raft_consensus.cc:359] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.906570  8454 ts_tablet_manager.cc:1403] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 12:59:09.906708  8453 raft_consensus.cc:385] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.906893  8453 raft_consensus.cc:740] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.907406  8453 consensus_queue.cc:260] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.909027  8453 ts_tablet_manager.cc:1434] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:59:09.909024  8454 raft_consensus.cc:359] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.909880  8454 raft_consensus.cc:385] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.910041  8453 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.910261  8454 raft_consensus.cc:740] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.911026  8454 consensus_queue.cc:260] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.913192  8454 ts_tablet_manager.cc:1434] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:09.914054  8454 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.915966  8453 tablet_bootstrap.cc:654] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.919691  8454 tablet_bootstrap.cc:654] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.922461  8458 raft_consensus.cc:493] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.923076  8458 raft_consensus.cc:515] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.926697  8458 leader_election.cc:290] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.933004  8453 tablet_bootstrap.cc:492] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.933446  8453 ts_tablet_manager.cc:1403] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.012s
I20260709 12:59:09.936143  8453 raft_consensus.cc:359] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.936771  8453 raft_consensus.cc:385] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.937041  8453 raft_consensus.cc:740] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.937651  8460 raft_consensus.cc:493] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.938150  8454 tablet_bootstrap.cc:492] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.938107  8460 raft_consensus.cc:515] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.938720  8454 ts_tablet_manager.cc:1403] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.011s
I20260709 12:59:09.937839  8453 consensus_queue.cc:260] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.940274  8460 leader_election.cc:290] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.941490  8453 ts_tablet_manager.cc:1434] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:09.941560  8454 raft_consensus.cc:359] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.942377  8454 raft_consensus.cc:385] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.942704  8454 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.942399  8453 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: Bootstrap starting.
I20260709 12:59:09.942821  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5907404a3eb544b0882ee5210cf13709" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:09.943570  8454 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.945600  8454 ts_tablet_manager.cc:1434] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:09.943897  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5907404a3eb544b0882ee5210cf13709" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:09.948096  8413 raft_consensus.cc:2468] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 0.
I20260709 12:59:09.948470  8453 tablet_bootstrap.cc:654] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.949187  8454 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: Bootstrap starting.
I20260709 12:59:09.949265  8230 leader_election.cc:304] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:09.950421  8489 raft_consensus.cc:2804] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.950799  8489 raft_consensus.cc:493] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.951164  8489 raft_consensus.cc:3060] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.943571  8339 raft_consensus.cc:2468] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 0.
I20260709 12:59:09.957525  8454 tablet_bootstrap.cc:654] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.960542  8489 raft_consensus.cc:515] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.961243  8453 tablet_bootstrap.cc:492] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: No bootstrap required, opened a new log
I20260709 12:59:09.961670  8453 ts_tablet_manager.cc:1403] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 12:59:09.962257  8454 tablet_bootstrap.cc:492] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: No bootstrap required, opened a new log
I20260709 12:59:09.962670  8454 ts_tablet_manager.cc:1403] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:59:09.963348  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:09.964260  8453 raft_consensus.cc:359] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.965052  8453 raft_consensus.cc:385] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.964982  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5907404a3eb544b0882ee5210cf13709" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:09.965411  8453 raft_consensus.cc:740] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.965670  8339 raft_consensus.cc:3060] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.965775  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:09.966051  8453 consensus_queue.cc:260] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.966395  8263 raft_consensus.cc:2468] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 0.
W20260709 12:59:09.967892  8305 leader_election.cc:343] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.968801  8305 leader_election.cc:304] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:09.970214  8453 ts_tablet_manager.cc:1434] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:59:09.971084  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5907404a3eb544b0882ee5210cf13709" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:09.971632  8413 raft_consensus.cc:3060] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.973563  8454 raft_consensus.cc:359] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.974211  8454 raft_consensus.cc:385] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.974454  8454 raft_consensus.cc:740] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.975081  8454 consensus_queue.cc:260] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:09.976981  8454 ts_tablet_manager.cc:1434] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 12:59:09.977826  8339 raft_consensus.cc:2468] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 1.
I20260709 12:59:09.978657  8413 raft_consensus.cc:2468] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 1.
I20260709 12:59:09.978886  8228 leader_election.cc:304] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:09.979884  8460 raft_consensus.cc:2804] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.980297  8460 raft_consensus.cc:697] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:09.981436  8489 leader_election.cc:290] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 1 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.981078  8460 consensus_queue.cc:237] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.982282  8458 raft_consensus.cc:2804] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.982656  8458 raft_consensus.cc:493] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.982963  8458 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.987910  8458 raft_consensus.cc:515] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:09.989672  8458 leader_election.cc:290] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:09.990324  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:09.990536  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:09.991057  8413 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.991022  8263 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.995316  8413 raft_consensus.cc:2468] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:09.996344  8305 leader_election.cc:304] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:09.997207  8458 raft_consensus.cc:2804] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.999177  8263 raft_consensus.cc:2468] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:09.999343  8458 raft_consensus.cc:697] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.000248  8458 consensus_queue.cc:237] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.002236  8509 raft_consensus.cc:493] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:10.002336  8159 catalog_manager.cc:5697] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.002902  8509 raft_consensus.cc:515] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.005992  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:10.006596  8263 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 0.
I20260709 12:59:10.007696  8305 leader_election.cc:304] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.008395  8458 raft_consensus.cc:2804] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:10.008698  8458 raft_consensus.cc:493] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:10.008968  8458 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.010553  8161 catalog_manager.cc:5697] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.012055  8509 leader_election.cc:290] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.013101  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:10.013835  8413 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 0.
I20260709 12:59:10.014029  8481 raft_consensus.cc:493] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:10.014505  8481 raft_consensus.cc:515] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.016444  8481 leader_election.cc:290] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.016222  8458 raft_consensus.cc:515] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.017462  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98dc7c8003d499f95f18cbe168ce755" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:10.017900  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98dc7c8003d499f95f18cbe168ce755" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:10.018059  8263 raft_consensus.cc:2468] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:10.018196  8458 leader_election.cc:290] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.018469  8339 raft_consensus.cc:2468] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:10.019380  8379 leader_election.cc:304] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:10.020152  8481 raft_consensus.cc:2804] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:10.020488  8481 raft_consensus.cc:493] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:10.020797  8481 raft_consensus.cc:3060] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.021381  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.021742  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.021904  8263 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.022276  8413 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.026494  8481 raft_consensus.cc:515] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.028000  8481 leader_election.cc:290] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.028744  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98dc7c8003d499f95f18cbe168ce755" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.029050  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98dc7c8003d499f95f18cbe168ce755" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:10.029460  8263 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:10.029524  8262 raft_consensus.cc:3060] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.029635  8339 raft_consensus.cc:3060] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.033548  8413 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:10.033630  8305 leader_election.cc:304] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.034638  8458 raft_consensus.cc:2804] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:10.034932  8262 raft_consensus.cc:2468] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:10.035125  8458 raft_consensus.cc:697] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.036031  8379 leader_election.cc:304] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:10.036064  8458 consensus_queue.cc:237] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.036774  8481 raft_consensus.cc:2804] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:10.036804  8339 raft_consensus.cc:2468] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:10.037398  8481 raft_consensus.cc:697] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:10.038229  8481 consensus_queue.cc:237] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.044482  8161 catalog_manager.cc:5697] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.046600  8159 catalog_manager.cc:5697] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 1 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:10.049489  8458 raft_consensus.cc:493] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:10.049966  8458 raft_consensus.cc:515] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.051899  8458 leader_election.cc:290] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.053802  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96a288a3eff4c3485e89b6f6f92507b" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:10.054409  8262 raft_consensus.cc:2468] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 0.
I20260709 12:59:10.055603  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96a288a3eff4c3485e89b6f6f92507b" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a" is_pre_election: true
I20260709 12:59:10.055939  8305 leader_election.cc:304] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.056229  8413 raft_consensus.cc:2468] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 0.
I20260709 12:59:10.056645  8458 raft_consensus.cc:2804] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:10.057021  8458 raft_consensus.cc:493] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:10.057480  8458 raft_consensus.cc:3060] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.062063  8458 raft_consensus.cc:515] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.063544  8458 leader_election.cc:290] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.064215  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96a288a3eff4c3485e89b6f6f92507b" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.064462  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96a288a3eff4c3485e89b6f6f92507b" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.064657  8262 raft_consensus.cc:3060] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.064973  8413 raft_consensus.cc:3060] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.069079  8413 raft_consensus.cc:2468] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:10.069089  8262 raft_consensus.cc:2468] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 1.
I20260709 12:59:10.070094  8305 leader_election.cc:304] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.070693  8458 raft_consensus.cc:2804] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:10.071096  8458 raft_consensus.cc:697] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 1 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.071763  8458 consensus_queue.cc:237] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.078579  8159 catalog_manager.cc:5697] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.088621  8413 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.094252  8460 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.104852  8489 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.179876  8481 raft_consensus.cc:493] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:10.180368  8481 raft_consensus.cc:515] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.182240  8481 leader_election.cc:290] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.183005  8262 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.183209  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:10.183362  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:10.183888  8263 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:10.183998  8339 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 0.
I20260709 12:59:10.184897  8379 leader_election.cc:304] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:10.185604  8481 raft_consensus.cc:2804] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:10.185952  8481 raft_consensus.cc:493] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:10.186275  8481 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.189563  8491 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.190941  8481 raft_consensus.cc:515] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.193423  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.193774  8339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:10.194039  8263 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.194341  8339 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.195358  8481 leader_election.cc:290] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.200770  8491 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.201213  8263 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:10.202225  8379 leader_election.cc:304] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:10.203172  8491 raft_consensus.cc:2804] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:10.203624  8491 raft_consensus.cc:697] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:10.204393  8491 consensus_queue.cc:237] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.205665  8339 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 1.
I20260709 12:59:10.215819  8159 catalog_manager.cc:5697] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 1 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:10.280161  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:39032
I20260709 12:59:10.280659  8413 raft_consensus.cc:606] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:10.295420  8338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "b1a612823877416488065c9c19319105"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:44612
I20260709 12:59:10.295956  8338 raft_consensus.cc:606] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:10.302832  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:39032
I20260709 12:59:10.303320  8413 raft_consensus.cc:606] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:10.309994  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530acf16d6842098206f7555680e87d"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
mode: GRACEFUL
new_leader_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:39032
I20260709 12:59:10.310487  8413 raft_consensus.cc:606] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Received request to transfer leadership to TS b1a612823877416488065c9c19319105
I20260709 12:59:10.322093  8263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:59:10.322548  8263 raft_consensus.cc:606] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Received request to transfer leadership to TS b1a612823877416488065c9c19319105
I20260709 12:59:10.402298  8464 raft_consensus.cc:493] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:10.402799  8464 raft_consensus.cc:515] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.404908  8464 leader_election.cc:290] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.405762  8489 consensus_queue.cc:1048] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.406447  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a612823877416488065c9c19319105" is_pre_election: true
I20260709 12:59:10.406706  8460 consensus_queue.cc:1048] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.407219  8338 raft_consensus.cc:2393] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84e1de3cad67432bb32985503a774c2a in current term 1: Already voted for candidate 4625def2990b484e9b6312f2721dcbd8 in this term.
I20260709 12:59:10.405680  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4625def2990b484e9b6312f2721dcbd8" is_pre_election: true
I20260709 12:59:10.408735  8379 leader_election.cc:304] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a; no voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105
I20260709 12:59:10.409515  8464 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:10.410224  8491 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.419946  8481 raft_consensus.cc:993] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.420464  8491 raft_consensus.cc:1081] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.423362  8464 raft_consensus.cc:2749] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:10.423869  8263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40422
I20260709 12:59:10.424414  8263 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.424825  8263 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.425498  8491 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.434306  8462 consensus_queue.cc:1048] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.437929  8263 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.435978  8460 consensus_queue.cc:1048] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:10.439186  8262 raft_consensus.cc:1240] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Rejecting Update request from peer 84e1de3cad67432bb32985503a774c2a for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:10.441452  8481 consensus_queue.cc:1059] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.442189  8481 raft_consensus.cc:3055] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.442483  8481 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:10.443259  8481 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.444133  8481 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.445629  8263 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.445853  8411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.449831  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:10.462206  8411 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:10.464941  8230 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: b1a612823877416488065c9c19319105
I20260709 12:59:10.466346  8527 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.466846  8527 raft_consensus.cc:697] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:10.468362  8527 consensus_queue.cc:237] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.483773  8159 catalog_manager.cc:5697] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 1 to 2, leader changed from 84e1de3cad67432bb32985503a774c2a (127.7.117.131) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.500430  8481 consensus_queue.cc:1048] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.503427  8458 consensus_queue.cc:1048] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.505093  8481 consensus_queue.cc:1048] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.505525  8458 consensus_queue.cc:1048] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.522889  8458 consensus_queue.cc:1048] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.537853  8458 consensus_queue.cc:1048] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.580448  8537 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.586966  8458 raft_consensus.cc:993] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.587450  8509 raft_consensus.cc:1081] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.589330  8264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40430
I20260709 12:59:10.590235  8264 raft_consensus.cc:493] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.590636  8264 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.594012  8537 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.597272  8264 raft_consensus.cc:515] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.599840  8264 leader_election.cc:290] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.600592  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:10.601235  8338 raft_consensus.cc:3055] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.601596  8338 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.602473  8338 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.603498  8338 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.605921  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.606501  8413 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.611038  8338 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:10.612190  8228 leader_election.cc:304] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.613000  8527 raft_consensus.cc:2804] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.613387  8413 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:10.613466  8527 raft_consensus.cc:697] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:10.614425  8527 consensus_queue.cc:237] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.623703  8159 catalog_manager.cc:5697] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 1 to 2, leader changed from b1a612823877416488065c9c19319105 (127.7.117.130) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.728960  8491 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.734616  8527 raft_consensus.cc:993] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:10.735085  8527 raft_consensus.cc:1081] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Signalling peer b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:10.737344  8464 raft_consensus.cc:993] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: : Instructing follower b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:10.737694  8338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:44592
I20260709 12:59:10.737882  8464 raft_consensus.cc:1081] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Signalling peer b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:10.738360  8338 raft_consensus.cc:493] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.738772  8338 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.741633  8337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6530acf16d6842098206f7555680e87d"
dest_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:44596
I20260709 12:59:10.742192  8337 raft_consensus.cc:493] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.742573  8337 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.748056  8464 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.749980  8337 raft_consensus.cc:515] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.749887  8338 raft_consensus.cc:515] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.753876  8338 leader_election.cc:290] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 2 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.753907  8337 leader_election.cc:290] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 2 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:10.756394  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.756989  8263 raft_consensus.cc:3055] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.757347  8263 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:10.757884  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:10.757893  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.758486  8262 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.758231  8263 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.759370  8412 raft_consensus.cc:3055] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.759799  8412 raft_consensus.cc:740] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:10.760102  8263 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.760667  8412 consensus_queue.cc:260] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.758270  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.761911  8413 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.765815  8262 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 2.
I20260709 12:59:10.766928  8263 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 2.
I20260709 12:59:10.767107  8305 leader_election.cc:304] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.767751  8412 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.768213  8305 leader_election.cc:304] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.769095  8509 raft_consensus.cc:2804] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.769621  8509 raft_consensus.cc:697] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.770522  8509 consensus_queue.cc:237] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.771567  8413 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 2.
I20260709 12:59:10.772607  8458 raft_consensus.cc:2804] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.773180  8458 raft_consensus.cc:697] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:10.774097  8458 consensus_queue.cc:237] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:10.775346  8412 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 2.
I20260709 12:59:10.781023  8160 catalog_manager.cc:5697] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 reported cstate change: term changed from 1 to 2, leader changed from 84e1de3cad67432bb32985503a774c2a (127.7.117.131) to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.793970  8160 catalog_manager.cc:5697] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 reported cstate change: term changed from 1 to 2, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.819878  8491 raft_consensus.cc:993] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.820369  8491 raft_consensus.cc:1081] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:10.822100  8263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40422
I20260709 12:59:10.822587  8263 raft_consensus.cc:493] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.822916  8263 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.827735  8263 raft_consensus.cc:515] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:10.829226  8263 leader_election.cc:290] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Requested vote from peers 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:10.830253  8412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:10.830361  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:10.830961  8337 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.830875  8412 raft_consensus.cc:3055] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.831488  8412 raft_consensus.cc:740] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:10.832106  8412 consensus_queue.cc:260] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:10.833241  8412 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.835758  8337 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:10.837049  8228 leader_election.cc:304] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:10.837868  8460 raft_consensus.cc:2804] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.838354  8460 raft_consensus.cc:697] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:10.838375  8412 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:10.839270  8460 consensus_queue.cc:237] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:10.845633  8160 catalog_manager.cc:5697] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 1 to 2, leader changed from 84e1de3cad67432bb32985503a774c2a (127.7.117.131) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.957428  8412 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.958689  8460 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.967445  8337 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.975204  8337 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.976634  8527 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.163945  8337 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.165484  8527 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.173951  8412 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.175894  8462 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.216974  8263 raft_consensus.cc:1275] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.218348  8509 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.226799  8412 raft_consensus.cc:1275] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.228196  8509 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.241933  8263 raft_consensus.cc:1275] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.243165  8458 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.252285  8412 raft_consensus.cc:1275] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.253927  8537 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.302783  8412 raft_consensus.cc:1275] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.304042  8462 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.312117  8337 raft_consensus.cc:1275] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.313748  8462 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:13.357120  7638 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 12:59:13.357856  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:13.372535  8263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:13.373059  8263 raft_consensus.cc:606] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Received request to transfer leadership to TS 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:13.375247  8263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:13.375721  8263 raft_consensus.cc:606] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Received request to transfer leadership to TS 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:13.376570  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:13.376994  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:13.712697  8527 raft_consensus.cc:993] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:13.713346  8552 raft_consensus.cc:1081] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Signalling peer 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:13.715092  8412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:39028
I20260709 12:59:13.715626  8412 raft_consensus.cc:493] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:13.715939  8412 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.720343  8552 raft_consensus.cc:993] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:13.720808  8527 raft_consensus.cc:1081] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Signalling peer 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:13.721755  8412 raft_consensus.cc:515] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.722611  8413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:39028
I20260709 12:59:13.723089  8413 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:13.723402  8413 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.723733  8412 leader_election.cc:290] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:13.724395  8263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:13.724970  8263 raft_consensus.cc:3055] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:13.725281  8263 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:13.725278  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:13.725932  8337 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.726251  8263 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.727455  8263 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.728554  8413 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.730927  8413 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:13.731261  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:13.731779  8262 raft_consensus.cc:3055] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:13.732089  8262 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:13.732719  8262 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.733004  8337 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:13.733935  8262 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.734014  8263 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:13.734316  8377 leader_election.cc:304] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:13.735127  8491 raft_consensus.cc:2804] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:13.735282  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:13.735883  8337 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.736254  8491 raft_consensus.cc:697] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:13.737004  8491 consensus_queue.cc:237] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.742130  8262 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:13.742293  8337 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:13.743185  8379 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:13.743870  8491 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:13.744354  8491 raft_consensus.cc:697] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:13.744441  8159 catalog_manager.cc:5697] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 2 to 3, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 3 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:13.745160  8491 consensus_queue.cc:237] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:13.759630  8159 catalog_manager.cc:5697] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 2 to 3, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 3 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:14.271839  8337 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.273257  8491 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.282682  8262 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.284703  8491 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.341216  8337 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.342410  8561 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.350432  8262 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.351893  8561 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.377713  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:14.391456  8413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:39032
I20260709 12:59:14.391932  8413 raft_consensus.cc:606] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:14.399204  8412 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:39032
I20260709 12:59:14.399782  8412 raft_consensus.cc:606] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:14.404667  8338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1"
dest_uuid: "b1a612823877416488065c9c19319105"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:44612
I20260709 12:59:14.405223  8338 raft_consensus.cc:606] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:14.409708  8338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530acf16d6842098206f7555680e87d"
dest_uuid: "b1a612823877416488065c9c19319105"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:44612
I20260709 12:59:14.410181  8338 raft_consensus.cc:606] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:14.414860  8338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "b1a612823877416488065c9c19319105"
mode: GRACEFUL
new_leader_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:44612
I20260709 12:59:14.415488  8338 raft_consensus.cc:606] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 LEADER]: Received request to transfer leadership to TS 4625def2990b484e9b6312f2721dcbd8
I20260709 12:59:14.555970  8549 raft_consensus.cc:993] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.556429  8570 raft_consensus.cc:1081] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.558033  8262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40430
I20260709 12:59:14.558574  8262 raft_consensus.cc:493] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.558887  8262 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.563333  8262 raft_consensus.cc:515] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.564981  8262 leader_election.cc:290] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:14.565762  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:14.566035  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcfa08578c0148ba866e24ff6dce61c1" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:14.566376  8338 raft_consensus.cc:3055] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:14.566555  8413 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.566744  8338 raft_consensus.cc:740] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:14.567622  8338 consensus_queue.cc:260] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.568813  8338 raft_consensus.cc:3060] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.573223  8413 raft_consensus.cc:2468] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:14.574440  8230 leader_election.cc:304] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:14.575088  8338 raft_consensus.cc:2468] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 2.
I20260709 12:59:14.575162  8527 raft_consensus.cc:2804] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:14.575649  8527 raft_consensus.cc:697] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:14.576493  8527 consensus_queue.cc:237] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.583029  8160 catalog_manager.cc:5697] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 1 to 2, leader changed from b1a612823877416488065c9c19319105 (127.7.117.130) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.596222  8549 raft_consensus.cc:993] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.596715  8570 raft_consensus.cc:1081] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.598249  8262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40430
I20260709 12:59:14.598722  8262 raft_consensus.cc:493] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.599030  8262 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.603580  8262 raft_consensus.cc:515] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.605134  8262 leader_election.cc:290] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 3 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:14.606158  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:14.606158  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:14.606956  8413 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.607029  8338 raft_consensus.cc:3055] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:14.607419  8338 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:14.608273  8338 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.609562  8338 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.613878  8413 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 3.
I20260709 12:59:14.615077  8230 leader_election.cc:304] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:14.615829  8527 raft_consensus.cc:2804] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:14.616214  8527 raft_consensus.cc:697] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:14.616391  8338 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 3.
I20260709 12:59:14.616950  8527 consensus_queue.cc:237] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.623502  8160 catalog_manager.cc:5697] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 2 to 3, leader changed from b1a612823877416488065c9c19319105 (127.7.117.130) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 3 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.777197  8549 raft_consensus.cc:993] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.777640  8570 raft_consensus.cc:1081] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.779075  8262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6530acf16d6842098206f7555680e87d"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40430
I20260709 12:59:14.779500  8262 raft_consensus.cc:493] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.779750  8262 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.784216  8262 raft_consensus.cc:515] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.785827  8262 leader_election.cc:290] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 3 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:14.786680  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:14.786830  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530acf16d6842098206f7555680e87d" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:14.787164  8338 raft_consensus.cc:3055] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:14.787348  8413 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.787586  8338 raft_consensus.cc:740] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:14.788403  8338 consensus_queue.cc:260] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.789371  8338 raft_consensus.cc:3060] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.791771  8413 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 3.
I20260709 12:59:14.792727  8230 leader_election.cc:304] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, 84e1de3cad67432bb32985503a774c2a; no voters: 
I20260709 12:59:14.793431  8527 raft_consensus.cc:2804] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:14.793824  8527 raft_consensus.cc:697] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:14.794564  8527 consensus_queue.cc:237] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.795053  8338 raft_consensus.cc:2468] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 3.
I20260709 12:59:14.801318  8160 catalog_manager.cc:5697] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 2 to 3, leader changed from b1a612823877416488065c9c19319105 (127.7.117.130) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 3 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.844579  8491 raft_consensus.cc:993] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.845060  8561 raft_consensus.cc:1081] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.846535  8262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40422
I20260709 12:59:14.847013  8262 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.847340  8262 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.854876  8262 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.856596  8262 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 4 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:14.857345  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:14.857532  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:14.857962  8338 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.858125  8413 raft_consensus.cc:3055] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:14.858505  8413 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:14.859134  8413 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.859937  8413 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.862736  8338 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 4.
I20260709 12:59:14.863770  8413 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 4.
I20260709 12:59:14.863723  8228 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:14.864488  8527 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:14.864938  8527 raft_consensus.cc:697] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:14.865618  8527 consensus_queue.cc:237] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.871219  8160 catalog_manager.cc:5697] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 3 to 4, leader changed from 84e1de3cad67432bb32985503a774c2a (127.7.117.131) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 4 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.926782  8561 raft_consensus.cc:993] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: : Instructing follower 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.927268  8563 raft_consensus.cc:1081] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Signalling peer 4625def2990b484e9b6312f2721dcbd8 to start an election
I20260709 12:59:14.928681  8262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
 from {username='slave'} at 127.0.0.1:40422
I20260709 12:59:14.929241  8262 raft_consensus.cc:493] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.929626  8262 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.934477  8262 raft_consensus.cc:515] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.936024  8262 leader_election.cc:290] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 4 election: Requested vote from peers b1a612823877416488065c9c19319105 (127.7.117.130:35243), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:14.936857  8338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:14.936987  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88643dfe8014f518aa2b0163d2ce48e" candidate_uuid: "4625def2990b484e9b6312f2721dcbd8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:14.937510  8413 raft_consensus.cc:3055] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:14.937487  8338 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.937958  8413 raft_consensus.cc:740] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:14.938681  8413 consensus_queue.cc:260] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.939734  8413 raft_consensus.cc:3060] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.944059  8338 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 4.
I20260709 12:59:14.945088  8228 leader_election.cc:304] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4625def2990b484e9b6312f2721dcbd8, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:14.945880  8569 raft_consensus.cc:2804] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:14.945878  8413 raft_consensus.cc:2468] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4625def2990b484e9b6312f2721dcbd8 in term 4.
I20260709 12:59:14.946501  8569 raft_consensus.cc:697] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Becoming Leader. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:14.947304  8569 consensus_queue.cc:237] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:14.952914  8160 catalog_manager.cc:5697] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 reported cstate change: term changed from 3 to 4, leader changed from 84e1de3cad67432bb32985503a774c2a (127.7.117.131) to 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129). New cstate: current_term: 4 leader_uuid: "4625def2990b484e9b6312f2721dcbd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:15.011081  8337 raft_consensus.cc:1275] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.012495  8577 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.024329  8413 raft_consensus.cc:1275] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.025703  8569 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.112852  8337 raft_consensus.cc:1275] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:15.114073  8569 consensus_queue.cc:1048] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:15.123656  8413 raft_consensus.cc:1275] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:15.124951  8527 consensus_queue.cc:1048] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:15.277853  8337 raft_consensus.cc:1275] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.279227  8527 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.287283  8413 raft_consensus.cc:1275] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.289474  8527 consensus_queue.cc:1048] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.298744  8413 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:15.299927  8527 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:15.307762  8337 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:15.309625  8577 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:15.371721  8413 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:15.372812  8527 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:15.380872  8337 raft_consensus.cc:1275] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Refusing update from remote peer 4625def2990b484e9b6312f2721dcbd8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:15.382239  8588 consensus_queue.cc:1048] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:17.422504  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:17.429939  8262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:17.430640  8262 raft_consensus.cc:606] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Received request to transfer leadership to TS b1a612823877416488065c9c19319105
I20260709 12:59:17.433091  8262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:17.433537  8262 raft_consensus.cc:606] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Received request to transfer leadership to TS 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:17.435550  8262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:17.435966  8262 raft_consensus.cc:606] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Received request to transfer leadership to TS 84e1de3cad67432bb32985503a774c2a
I20260709 12:59:17.438129  8262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
mode: GRACEFUL
new_leader_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:57474
I20260709 12:59:17.438541  8262 raft_consensus.cc:606] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Received request to transfer leadership to TS b1a612823877416488065c9c19319105
I20260709 12:59:17.439450  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:17.439941  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:17.606616  8527 raft_consensus.cc:993] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:17.607098  8593 raft_consensus.cc:1081] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Signalling peer 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:17.608644  8413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:39028
I20260709 12:59:17.609104  8413 raft_consensus.cc:493] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.609452  8413 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.614640  8413 raft_consensus.cc:515] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:17.616369  8413 leader_election.cc:290] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:17.617144  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:17.617942  8262 raft_consensus.cc:3055] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:17.618120  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b27a76e4da413595794b2b7b24fc56" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:17.618269  8262 raft_consensus.cc:740] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:17.618688  8337 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.619200  8262 consensus_queue.cc:260] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:17.620339  8262 raft_consensus.cc:3060] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.623412  8337 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:17.624519  8377 leader_election.cc:304] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:17.625134  8262 raft_consensus.cc:2468] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 3.
I20260709 12:59:17.625504  8563 raft_consensus.cc:2804] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:17.626822  8563 raft_consensus.cc:697] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:17.627800  8563 consensus_queue.cc:237] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } }
I20260709 12:59:17.634877  8159 catalog_manager.cc:5697] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 2 to 3, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 3 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.773624  8527 raft_consensus.cc:993] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:17.774080  8593 raft_consensus.cc:1081] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Signalling peer 84e1de3cad67432bb32985503a774c2a to start an election
I20260709 12:59:17.776017  8413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db"
dest_uuid: "84e1de3cad67432bb32985503a774c2a"
 from {username='slave'} at 127.0.0.1:39028
I20260709 12:59:17.776515  8413 raft_consensus.cc:493] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.776849  8413 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:17.783790  8413 raft_consensus.cc:515] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.785784  8413 leader_election.cc:290] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 2 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), b1a612823877416488065c9c19319105 (127.7.117.130:35243)
I20260709 12:59:17.786860  8337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1a612823877416488065c9c19319105"
I20260709 12:59:17.786931  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a46307364d724a5f90fe7b029b0746db" candidate_uuid: "84e1de3cad67432bb32985503a774c2a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:17.787525  8337 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:17.787696  8262 raft_consensus.cc:3055] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:17.788059  8262 raft_consensus.cc:740] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:17.788705  8262 consensus_queue.cc:260] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.790076  8262 raft_consensus.cc:3060] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:17.793529  8337 raft_consensus.cc:2468] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 2.
I20260709 12:59:17.794241  8262 raft_consensus.cc:2468] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e1de3cad67432bb32985503a774c2a in term 2.
I20260709 12:59:17.794476  8377 leader_election.cc:304] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:17.795396  8563 raft_consensus.cc:2804] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:17.795871  8563 raft_consensus.cc:697] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 2 LEADER]: Becoming Leader. State: Replica: 84e1de3cad67432bb32985503a774c2a, State: Running, Role: LEADER
I20260709 12:59:17.796612  8563 consensus_queue.cc:237] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.804957  8159 catalog_manager.cc:5697] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a reported cstate change: term changed from 1 to 2, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to 84e1de3cad67432bb32985503a774c2a (127.7.117.131). New cstate: current_term: 2 leader_uuid: "84e1de3cad67432bb32985503a774c2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:17.810956  8527 raft_consensus.cc:993] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:17.811455  8593 raft_consensus.cc:1081] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Signalling peer b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:17.812984  8337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc"
dest_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:44592
I20260709 12:59:17.813540  8337 raft_consensus.cc:493] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.813894  8337 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:17.821493  8337 raft_consensus.cc:515] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.823757  8337 leader_election.cc:290] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 5 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:17.824333  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:17.824896  8262 raft_consensus.cc:3055] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:59:17.824782  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9cbdc6d795433ab33c816e5e7552fc" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:17.825186  8262 raft_consensus.cc:740] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:17.825337  8413 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:17.826015  8262 consensus_queue.cc:260] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.827024  8262 raft_consensus.cc:3060] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:17.831575  8413 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 5.
I20260709 12:59:17.832841  8305 leader_election.cc:304] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:17.833716  8570 raft_consensus.cc:2804] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:59:17.833740  8262 raft_consensus.cc:2468] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 5.
I20260709 12:59:17.835121  8570 raft_consensus.cc:697] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 5 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:17.836033  8570 consensus_queue.cc:237] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.842336  8159 catalog_manager.cc:5697] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 reported cstate change: term changed from 4 to 5, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 5 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.948737  8527 raft_consensus.cc:993] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: : Instructing follower b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:17.949172  8593 raft_consensus.cc:1081] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Signalling peer b1a612823877416488065c9c19319105 to start an election
I20260709 12:59:17.950640  8337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10"
dest_uuid: "b1a612823877416488065c9c19319105"
 from {username='slave'} at 127.0.0.1:44592
I20260709 12:59:17.951077  8337 raft_consensus.cc:493] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.951341  8337 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.955315  8337 raft_consensus.cc:515] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.956676  8337 leader_election.cc:290] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 4 election: Requested vote from peers 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167), 84e1de3cad67432bb32985503a774c2a (127.7.117.131:43703)
I20260709 12:59:17.957379  8262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4625def2990b484e9b6312f2721dcbd8"
I20260709 12:59:17.957683  8413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7f285279264df0bd977ef1ecddef10" candidate_uuid: "b1a612823877416488065c9c19319105" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "84e1de3cad67432bb32985503a774c2a"
I20260709 12:59:17.957937  8262 raft_consensus.cc:3055] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:17.958236  8413 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.958231  8262 raft_consensus.cc:740] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4625def2990b484e9b6312f2721dcbd8, State: Running, Role: LEADER
I20260709 12:59:17.959163  8262 consensus_queue.cc:260] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.960166  8262 raft_consensus.cc:3060] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.963197  8413 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 4.
I20260709 12:59:17.964027  8305 leader_election.cc:304] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e1de3cad67432bb32985503a774c2a, b1a612823877416488065c9c19319105; no voters: 
I20260709 12:59:17.964727  8570 raft_consensus.cc:2804] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:17.964743  8262 raft_consensus.cc:2468] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a612823877416488065c9c19319105 in term 4.
I20260709 12:59:17.965263  8570 raft_consensus.cc:697] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 4 LEADER]: Becoming Leader. State: Replica: b1a612823877416488065c9c19319105, State: Running, Role: LEADER
I20260709 12:59:17.966473  8570 consensus_queue.cc:237] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } }
I20260709 12:59:17.972872  8159 catalog_manager.cc:5697] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 reported cstate change: term changed from 3 to 4, leader changed from 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129) to b1a612823877416488065c9c19319105 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "b1a612823877416488065c9c19319105" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:18.042497  8337 raft_consensus.cc:1275] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:18.043737  8563 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:18.053120  8262 raft_consensus.cc:1275] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:18.054720  8606 consensus_queue.cc:1048] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:18.234899  8262 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 5 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:18.235952  8570 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:18.246593  8413 raft_consensus.cc:1275] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 5 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:18.249694  8612 consensus_queue.cc:1048] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:18.269747  8337 raft_consensus.cc:1275] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:18.270764  8606 consensus_queue.cc:1048] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a612823877416488065c9c19319105" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 35243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:18.279448  8262 raft_consensus.cc:1275] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Refusing update from remote peer 84e1de3cad67432bb32985503a774c2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:18.280740  8606 consensus_queue.cc:1048] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4625def2990b484e9b6312f2721dcbd8" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:18.440893  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:18.447407  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 12:59:18.469552  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.470347  7638 tablet_replica.cc:333] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.471225  7638 raft_consensus.cc:2243] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.472257  7638 raft_consensus.cc:2272] T a88643dfe8014f518aa2b0163d2ce48e P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.472666  8379 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 12:59:18.474797  7638 tablet_replica.cc:333] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.475590  7638 raft_consensus.cc:2243] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:18.475827  8379 consensus_peers.cc:597] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a -> Peer 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167): Couldn't send request to peer 4625def2990b484e9b6312f2721dcbd8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.476136  7638 raft_consensus.cc:2272] T e96a288a3eff4c3485e89b6f6f92507b P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.478534  7638 tablet_replica.cc:333] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.479022  7638 raft_consensus.cc:2243] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.479348  7638 raft_consensus.cc:2272] T 0a7f285279264df0bd977ef1ecddef10 P 4625def2990b484e9b6312f2721dcbd8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.480917  7638 tablet_replica.cc:333] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.481477  7638 raft_consensus.cc:2243] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.481805  7638 raft_consensus.cc:2272] T c98dc7c8003d499f95f18cbe168ce755 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.483371  7638 tablet_replica.cc:333] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.483842  7638 raft_consensus.cc:2243] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.484598  7638 raft_consensus.cc:2272] T fcfa08578c0148ba866e24ff6dce61c1 P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.486061  7638 tablet_replica.cc:333] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.486471  7638 raft_consensus.cc:2243] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.486783  7638 raft_consensus.cc:2272] T df9cbdc6d795433ab33c816e5e7552fc P 4625def2990b484e9b6312f2721dcbd8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.488726  7638 tablet_replica.cc:333] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.489372  7638 raft_consensus.cc:2243] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.490157  7638 raft_consensus.cc:2272] T 6530acf16d6842098206f7555680e87d P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.491719  7638 tablet_replica.cc:333] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.492113  7638 raft_consensus.cc:2243] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.492408  7638 raft_consensus.cc:2272] T a46307364d724a5f90fe7b029b0746db P 4625def2990b484e9b6312f2721dcbd8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.494240  7638 tablet_replica.cc:333] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.494642  7638 raft_consensus.cc:2243] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.494944  7638 raft_consensus.cc:2272] T d4b27a76e4da413595794b2b7b24fc56 P 4625def2990b484e9b6312f2721dcbd8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.496660  7638 tablet_replica.cc:333] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8: stopping tablet replica
I20260709 12:59:18.497208  7638 raft_consensus.cc:2243] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.498112  7638 raft_consensus.cc:2272] T 5907404a3eb544b0882ee5210cf13709 P 4625def2990b484e9b6312f2721dcbd8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.506270  8305 consensus_peers.cc:597] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 -> Peer 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167): Couldn't send request to peer 4625def2990b484e9b6312f2721dcbd8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.510031  8413 raft_consensus.cc:1275] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Refusing update from remote peer b1a612823877416488065c9c19319105: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:18.511472  8613 consensus_queue.cc:1048] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e1de3cad67432bb32985503a774c2a" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:18.522837  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
I20260709 12:59:18.542497  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
I20260709 12:59:18.563544  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.564342  7638 tablet_replica.cc:333] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.565328  7638 raft_consensus.cc:2243] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.565912  7638 raft_consensus.cc:2272] T c98dc7c8003d499f95f18cbe168ce755 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.567603  7638 tablet_replica.cc:333] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.568078  7638 raft_consensus.cc:2243] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.568897  7638 raft_consensus.cc:2272] T df9cbdc6d795433ab33c816e5e7552fc P b1a612823877416488065c9c19319105 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.570775  7638 tablet_replica.cc:333] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.571234  7638 raft_consensus.cc:2243] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.571595  7638 raft_consensus.cc:2272] T fcfa08578c0148ba866e24ff6dce61c1 P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.573045  7638 tablet_replica.cc:333] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.573546  7638 raft_consensus.cc:2243] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.573881  7638 raft_consensus.cc:2272] T 5907404a3eb544b0882ee5210cf13709 P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.575492  7638 tablet_replica.cc:333] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.575963  7638 raft_consensus.cc:2243] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.576295  7638 raft_consensus.cc:2272] T d4b27a76e4da413595794b2b7b24fc56 P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.578110  7638 tablet_replica.cc:333] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.578531  7638 raft_consensus.cc:2243] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.579146  7638 raft_consensus.cc:2272] T e96a288a3eff4c3485e89b6f6f92507b P b1a612823877416488065c9c19319105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.580579  7638 tablet_replica.cc:333] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.580981  7638 raft_consensus.cc:2243] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.581661  7638 raft_consensus.cc:2272] T 0a7f285279264df0bd977ef1ecddef10 P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.583359  7638 tablet_replica.cc:333] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.583837  7638 raft_consensus.cc:2243] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.584281  7638 raft_consensus.cc:2272] T a88643dfe8014f518aa2b0163d2ce48e P b1a612823877416488065c9c19319105 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.585999  7638 tablet_replica.cc:333] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.586403  7638 raft_consensus.cc:2243] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.586752  7638 raft_consensus.cc:2272] T 6530acf16d6842098206f7555680e87d P b1a612823877416488065c9c19319105 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.588267  7638 tablet_replica.cc:333] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105: stopping tablet replica
I20260709 12:59:18.588630  7638 raft_consensus.cc:2243] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.588943  7638 raft_consensus.cc:2272] T a46307364d724a5f90fe7b029b0746db P b1a612823877416488065c9c19319105 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.616428  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
W20260709 12:59:18.630527  8379 consensus_peers.cc:597] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a -> Peer 4625def2990b484e9b6312f2721dcbd8 (127.7.117.129:39167): Couldn't send request to peer 4625def2990b484e9b6312f2721dcbd8. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.637145  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
I20260709 12:59:18.656738  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.657459  7638 tablet_replica.cc:333] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.658011  7638 raft_consensus.cc:2243] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.658757  7638 raft_consensus.cc:2272] T a46307364d724a5f90fe7b029b0746db P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.661207  7638 tablet_replica.cc:333] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.661799  7638 raft_consensus.cc:2243] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.662240  7638 raft_consensus.cc:2272] T 6530acf16d6842098206f7555680e87d P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.663843  7638 tablet_replica.cc:333] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.664359  7638 raft_consensus.cc:2243] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.664767  7638 raft_consensus.cc:2272] T 0a7f285279264df0bd977ef1ecddef10 P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.666728  7638 tablet_replica.cc:333] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.667232  7638 raft_consensus.cc:2243] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.667622  7638 raft_consensus.cc:2272] T a88643dfe8014f518aa2b0163d2ce48e P 84e1de3cad67432bb32985503a774c2a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.669113  7638 tablet_replica.cc:333] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.669632  7638 raft_consensus.cc:2243] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.670023  7638 raft_consensus.cc:2272] T e96a288a3eff4c3485e89b6f6f92507b P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.671535  7638 tablet_replica.cc:333] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.671991  7638 raft_consensus.cc:2243] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.672672  7638 raft_consensus.cc:2272] T d4b27a76e4da413595794b2b7b24fc56 P 84e1de3cad67432bb32985503a774c2a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.674548  7638 tablet_replica.cc:333] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.675034  7638 raft_consensus.cc:2243] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.675407  7638 raft_consensus.cc:2272] T 5907404a3eb544b0882ee5210cf13709 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.676929  7638 tablet_replica.cc:333] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.677441  7638 raft_consensus.cc:2243] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.677805  7638 raft_consensus.cc:2272] T fcfa08578c0148ba866e24ff6dce61c1 P 84e1de3cad67432bb32985503a774c2a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.679217  7638 tablet_replica.cc:333] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.679638  7638 raft_consensus.cc:2243] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.680347  7638 raft_consensus.cc:2272] T c98dc7c8003d499f95f18cbe168ce755 P 84e1de3cad67432bb32985503a774c2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.681950  7638 tablet_replica.cc:333] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a: stopping tablet replica
I20260709 12:59:18.682410  7638 raft_consensus.cc:2243] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.682789  7638 raft_consensus.cc:2272] T df9cbdc6d795433ab33c816e5e7552fc P 84e1de3cad67432bb32985503a774c2a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.710868  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 12:59:18.729719  7638 master.cc:562] Master@127.7.117.190:46313 shutting down...
I20260709 12:59:18.750146  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.750661  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.750986  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0f38dfa3f712443d8e544ed87ff54b20: stopping tablet replica
I20260709 12:59:18.770977  7638 master.cc:584] Master@127.7.117.190:46313 shutdown complete.
4625def2990b484e9b6312f2721dcbd8  5
84e1de3cad67432bb32985503a774c2a  1
b1a612823877416488065c9c19319105  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10318 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:59:18.812605  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:45111
I20260709 12:59:18.813763  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:18.819476  8623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.819561  8622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.820405  8625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:18.822127  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:18.822887  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:18.823060  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:18.823179  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601958823160 us; error 0 us; skew 500 ppm
I20260709 12:59:18.823613  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:18.825899  7638 webserver.cc:533] Webserver started at http://127.7.117.190:43663/ using document root <none> and password file <none>
I20260709 12:59:18.826344  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:18.826496  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:18.826717  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:18.827744  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "5b6e9771cda34fa68ec7172c32ecd31b"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-ndzg"
I20260709 12:59:18.831919  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 12:59:18.835161  8630 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.835873  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:59:18.836202  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "5b6e9771cda34fa68ec7172c32ecd31b"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-ndzg"
I20260709 12:59:18.836479  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:18.853061  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:18.854393  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:18.892092  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:45111
I20260709 12:59:18.892180  8681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:45111 every 8 connection(s)
I20260709 12:59:18.896015  8682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:18.905524  8682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Bootstrap starting.
I20260709 12:59:18.909934  8682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.913980  8682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: No bootstrap required, opened a new log
I20260709 12:59:18.916461  8682 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER }
I20260709 12:59:18.916862  8682 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.917088  8682 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6e9771cda34fa68ec7172c32ecd31b, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.917797  8682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER }
I20260709 12:59:18.918252  8682 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:18.918480  8682 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:18.918746  8682 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.922830  8682 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER }
I20260709 12:59:18.923321  8682 leader_election.cc:304] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6e9771cda34fa68ec7172c32ecd31b; no voters: 
I20260709 12:59:18.924392  8682 leader_election.cc:290] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:18.924778  8685 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.926144  8685 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 1 LEADER]: Becoming Leader. State: Replica: 5b6e9771cda34fa68ec7172c32ecd31b, State: Running, Role: LEADER
I20260709 12:59:18.926715  8685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER }
I20260709 12:59:18.927260  8682 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:18.932011  8686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER } }
I20260709 12:59:18.932228  8687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6e9771cda34fa68ec7172c32ecd31b. Latest consensus state: current_term: 1 leader_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6e9771cda34fa68ec7172c32ecd31b" member_type: VOTER } }
I20260709 12:59:18.932683  8686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:18.932807  8687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:18.934893  8693 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:18.940215  8693 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:18.942382  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:18.948431  8693 catalog_manager.cc:1383] Generated new cluster ID: cfc77e6c19b146abb36a37fde71b2145
I20260709 12:59:18.948712  8693 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:18.978350  8693 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:18.979578  8693 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:18.996021  8693 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Generated new TSK 0
I20260709 12:59:18.996616  8693 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:19.008378  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:19.014094  8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.015866  8704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.017518  8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:19.017635  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:19.018642  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:19.018822  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:19.018945  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601959018934 us; error 0 us; skew 500 ppm
I20260709 12:59:19.019390  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:19.021634  7638 webserver.cc:533] Webserver started at http://127.7.117.129:38703/ using document root <none> and password file <none>
I20260709 12:59:19.022073  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:19.022236  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:19.022446  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:19.023432  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "b60591217ee24378bb66a86d6d0dda52"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.027709  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:59:19.031121  8711 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.031824  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:59:19.032105  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "b60591217ee24378bb66a86d6d0dda52"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.032394  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:19.065266  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:19.066526  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:19.067965  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:19.070163  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:19.070353  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.070604  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:19.070770  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.109453  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:39991
I20260709 12:59:19.109545  8773 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:39991 every 8 connection(s)
I20260709 12:59:19.114202  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:19.121721  8779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.122802  8778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.124573  8781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:19.124766  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:19.125847  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:19.126029  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:19.126258  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601959126236 us; error 0 us; skew 500 ppm
I20260709 12:59:19.126772  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:19.128468  8774 heartbeater.cc:344] Connected to a master server at 127.7.117.190:45111
I20260709 12:59:19.128790  8774 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:19.129499  7638 webserver.cc:533] Webserver started at http://127.7.117.130:33105/ using document root <none> and password file <none>
I20260709 12:59:19.129606  8774 heartbeater.cc:507] Master 127.7.117.190:45111 requested a full tablet report, sending...
I20260709 12:59:19.130045  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:19.130267  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:19.130553  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:19.131886  8647 ts_manager.cc:194] Registered new tserver with Master: b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:19.131935  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "29c7934ee528454785432732d01ed0a0"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.134022  8647 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55184
I20260709 12:59:19.137519  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:59:19.141036  8786 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.141834  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:59:19.142138  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "29c7934ee528454785432732d01ed0a0"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.142413  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:19.161969  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:19.163196  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:19.164641  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:19.166831  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:19.167059  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.167291  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:19.167454  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.205726  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:44185
I20260709 12:59:19.205830  8848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:44185 every 8 connection(s)
I20260709 12:59:19.210240  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:19.217118  8852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.218824  8853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.221048  8855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:19.221902  7638 server_base.cc:1061] running on GCE node
I20260709 12:59:19.222625  8849 heartbeater.cc:344] Connected to a master server at 127.7.117.190:45111
I20260709 12:59:19.222965  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:59:19.222975  8849 heartbeater.cc:461] Registering TS with master...
W20260709 12:59:19.223302  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:19.223517  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601959223500 us; error 0 us; skew 500 ppm
I20260709 12:59:19.223955  8849 heartbeater.cc:507] Master 127.7.117.190:45111 requested a full tablet report, sending...
I20260709 12:59:19.224066  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:19.226042  8647 ts_manager.cc:194] Registered new tserver with Master: 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:19.227015  7638 webserver.cc:533] Webserver started at http://127.7.117.131:42753/ using document root <none> and password file <none>
I20260709 12:59:19.227517  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:19.227524  8647 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55196
I20260709 12:59:19.227805  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:19.228194  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:19.229570  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "7fd110c312c14e17a2bbe6d453493cf2"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.233801  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:59:19.237579  8860 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.238361  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:59:19.238767  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "7fd110c312c14e17a2bbe6d453493cf2"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-ndzg"
I20260709 12:59:19.239040  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:19.254078  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:19.255164  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:19.256604  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:19.258811  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:19.258987  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.259276  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:19.259438  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.297981  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:44123
I20260709 12:59:19.298074  8922 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:44123 every 8 connection(s)
I20260709 12:59:19.311115  8923 heartbeater.cc:344] Connected to a master server at 127.7.117.190:45111
I20260709 12:59:19.311436  8923 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:19.312071  8923 heartbeater.cc:507] Master 127.7.117.190:45111 requested a full tablet report, sending...
I20260709 12:59:19.313838  8647 ts_manager.cc:194] Registered new tserver with Master: 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:19.314257  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013221194s
I20260709 12:59:19.315465  8647 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55202
I20260709 12:59:19.340102  8647 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55206:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:19.346306  8647 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:19.552130  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 1f2e6f6b24e84190b722a8e69bbe97fa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.553719  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2e6f6b24e84190b722a8e69bbe97fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.553478  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 739742728baa463f9d9f5e5884dd2e30 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.554879  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739742728baa463f9d9f5e5884dd2e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.555701  8739 tablet_service.cc:1511] Processing CreateTablet for tablet 44b3244bf12f42aba79421575a96c621 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.557106  8739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b3244bf12f42aba79421575a96c621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.557749  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 44b3244bf12f42aba79421575a96c621 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.558465  8735 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0adf8736244d3b208cbc85700f0f9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.559027  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b3244bf12f42aba79421575a96c621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.559736  8735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0adf8736244d3b208cbc85700f0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.562448  8734 tablet_service.cc:1511] Processing CreateTablet for tablet f091122e10604f27acd758d1f8da8081 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.563722  8734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f091122e10604f27acd758d1f8da8081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.565692  8736 tablet_service.cc:1511] Processing CreateTablet for tablet 42d038e54cdc4dfaa0cfec37a43480ee (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.566973  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d038e54cdc4dfaa0cfec37a43480ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.568397  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 1f2e6f6b24e84190b722a8e69bbe97fa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.569710  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2e6f6b24e84190b722a8e69bbe97fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.572754  8733 tablet_service.cc:1511] Processing CreateTablet for tablet de733f99bf86451da507373e7f4b78aa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.574131  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de733f99bf86451da507373e7f4b78aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.578878  8732 tablet_service.cc:1511] Processing CreateTablet for tablet 011b370821004185a003be210a525c01 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.579511  8880 tablet_service.cc:1511] Processing CreateTablet for tablet fe13736604454be3b391b8e946192961 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.580179  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011b370821004185a003be210a525c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.580243  8888 tablet_service.cc:1511] Processing CreateTablet for tablet 44b3244bf12f42aba79421575a96c621 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.580864  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe13736604454be3b391b8e946192961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.581638  8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b3244bf12f42aba79421575a96c621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.582854  8879 tablet_service.cc:1511] Processing CreateTablet for tablet 53864dd6ef544420915d557a63d5cd91 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.582845  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 739742728baa463f9d9f5e5884dd2e30 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.582845  8731 tablet_service.cc:1511] Processing CreateTablet for tablet fe13736604454be3b391b8e946192961 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.585632  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe13736604454be3b391b8e946192961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.585445  8730 tablet_service.cc:1511] Processing CreateTablet for tablet 53864dd6ef544420915d557a63d5cd91 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.584324  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53864dd6ef544420915d557a63d5cd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.586326  8887 tablet_service.cc:1511] Processing CreateTablet for tablet 1f2e6f6b24e84190b722a8e69bbe97fa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.586751  8730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53864dd6ef544420915d557a63d5cd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.587571  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2e6f6b24e84190b722a8e69bbe97fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.584331  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739742728baa463f9d9f5e5884dd2e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.590325  8886 tablet_service.cc:1511] Processing CreateTablet for tablet 739742728baa463f9d9f5e5884dd2e30 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.591550  8886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739742728baa463f9d9f5e5884dd2e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.594481  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 42d038e54cdc4dfaa0cfec37a43480ee (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.595781  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d038e54cdc4dfaa0cfec37a43480ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.598420  8884 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0adf8736244d3b208cbc85700f0f9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.599658  8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0adf8736244d3b208cbc85700f0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.602490  8883 tablet_service.cc:1511] Processing CreateTablet for tablet f091122e10604f27acd758d1f8da8081 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.603729  8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f091122e10604f27acd758d1f8da8081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.606335  8882 tablet_service.cc:1511] Processing CreateTablet for tablet de733f99bf86451da507373e7f4b78aa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.607561  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de733f99bf86451da507373e7f4b78aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.610426  8881 tablet_service.cc:1511] Processing CreateTablet for tablet 011b370821004185a003be210a525c01 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.610636  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 42d038e54cdc4dfaa0cfec37a43480ee (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.611964  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011b370821004185a003be210a525c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.612238  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d038e54cdc4dfaa0cfec37a43480ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.619777  8810 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0adf8736244d3b208cbc85700f0f9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.622478  8809 tablet_service.cc:1511] Processing CreateTablet for tablet f091122e10604f27acd758d1f8da8081 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.624756  8808 tablet_service.cc:1511] Processing CreateTablet for tablet de733f99bf86451da507373e7f4b78aa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.621155  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0adf8736244d3b208cbc85700f0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.632280  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 011b370821004185a003be210a525c01 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.634835  8806 tablet_service.cc:1511] Processing CreateTablet for tablet fe13736604454be3b391b8e946192961 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.643347  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f091122e10604f27acd758d1f8da8081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.646095  8805 tablet_service.cc:1511] Processing CreateTablet for tablet 53864dd6ef544420915d557a63d5cd91 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.647373  8805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53864dd6ef544420915d557a63d5cd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.671396  8735 tablet_service.cc:1511] Processing CreateTablet for tablet c24c6743859d43109fabeaf7ee8c8149 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.672688  8735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24c6743859d43109fabeaf7ee8c8149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.676378  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 108cbfcea5a2457d89f3cf70cf52b3ad (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.699306  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108cbfcea5a2457d89f3cf70cf52b3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.687556  8736 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91581ed67848b791865dfe11e4fc78 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.700613  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91581ed67848b791865dfe11e4fc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.689170  8813 tablet_service.cc:1511] Processing CreateTablet for tablet c24c6743859d43109fabeaf7ee8c8149 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.693671  8806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe13736604454be3b391b8e946192961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.713066  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de733f99bf86451da507373e7f4b78aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.713068  8882 tablet_service.cc:1511] Processing CreateTablet for tablet c24c6743859d43109fabeaf7ee8c8149 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.719350  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24c6743859d43109fabeaf7ee8c8149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.721555  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011b370821004185a003be210a525c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.723044  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24c6743859d43109fabeaf7ee8c8149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.732031  8814 tablet_service.cc:1511] Processing CreateTablet for tablet 108cbfcea5a2457d89f3cf70cf52b3ad (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.733398  8814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108cbfcea5a2457d89f3cf70cf52b3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.743018  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 108cbfcea5a2457d89f3cf70cf52b3ad (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.748909  8939 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:19.753767  8938 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:19.754797  8939 tablet_bootstrap.cc:654] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.755438  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108cbfcea5a2457d89f3cf70cf52b3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.756938  8882 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91581ed67848b791865dfe11e4fc78 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.759721  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91581ed67848b791865dfe11e4fc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.761112  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91581ed67848b791865dfe11e4fc78 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.762322  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91581ed67848b791865dfe11e4fc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.781272  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 1a0644aef66444b58e8633da6993535c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.781262  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 1a0644aef66444b58e8633da6993535c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.787240  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a0644aef66444b58e8633da6993535c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.787325  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a0644aef66444b58e8633da6993535c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.793913  8940 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:19.797475  8738 tablet_service.cc:1511] Processing CreateTablet for tablet db96d769069547e38edb50dbebdbe00a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.784745  8938 tablet_bootstrap.cc:654] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.798743  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db96d769069547e38edb50dbebdbe00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.800532  8882 tablet_service.cc:1511] Processing CreateTablet for tablet 1a0644aef66444b58e8633da6993535c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.803617  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a0644aef66444b58e8633da6993535c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.802582  8939 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:19.811830  8939 ts_tablet_manager.cc:1403] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.063s	user 0.020s	sys 0.011s
I20260709 12:59:19.814733  8939 raft_consensus.cc:359] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.815418  8939 raft_consensus.cc:385] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.815665  8939 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.816216  8939 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.820634  8940 tablet_bootstrap.cc:654] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.845248  8810 tablet_service.cc:1511] Processing CreateTablet for tablet db96d769069547e38edb50dbebdbe00a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.846698  8885 tablet_service.cc:1511] Processing CreateTablet for tablet db96d769069547e38edb50dbebdbe00a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.847915  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db96d769069547e38edb50dbebdbe00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.848701  8731 tablet_service.cc:1511] Processing CreateTablet for tablet c5bf6ec0e6a745d0967928a3361eb8d8 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.849959  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bf6ec0e6a745d0967928a3361eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.857767  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db96d769069547e38edb50dbebdbe00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.880126  8881 tablet_service.cc:1511] Processing CreateTablet for tablet c5bf6ec0e6a745d0967928a3361eb8d8 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.880609  8813 tablet_service.cc:1511] Processing CreateTablet for tablet c5bf6ec0e6a745d0967928a3361eb8d8 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.881825  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bf6ec0e6a745d0967928a3361eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.881918  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bf6ec0e6a745d0967928a3361eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.892684  8939 ts_tablet_manager.cc:1434] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.080s	user 0.022s	sys 0.017s
I20260709 12:59:19.893432  8774 heartbeater.cc:499] Master 127.7.117.190:45111 was elected leader, sending a full tablet report...
I20260709 12:59:19.896836  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 318340b5bf3b4640972d271299414f94 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.900190  8939 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:19.902107  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.903373  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.903461  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.904861  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.908145  8887 tablet_service.cc:1511] Processing CreateTablet for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.909533  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.910727  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 318340b5bf3b4640972d271299414f94 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.911983  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318340b5bf3b4640972d271299414f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.914803  8938 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:19.915256  8938 ts_tablet_manager.cc:1403] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.162s	user 0.012s	sys 0.010s
I20260709 12:59:19.918243  8938 raft_consensus.cc:359] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.919044  8938 raft_consensus.cc:385] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.919438  8938 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.921255  8730 tablet_service.cc:1511] Processing CreateTablet for tablet b476fe65810048fdb66dbeff393c8ccb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.922709  8730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b476fe65810048fdb66dbeff393c8ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.929505  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318340b5bf3b4640972d271299414f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.928414  8938 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.932664  8812 tablet_service.cc:1511] Processing CreateTablet for tablet b476fe65810048fdb66dbeff393c8ccb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.935084  8939 tablet_bootstrap.cc:654] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.933847  8880 tablet_service.cc:1511] Processing CreateTablet for tablet b476fe65810048fdb66dbeff393c8ccb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.935240  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b476fe65810048fdb66dbeff393c8ccb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:19.938215  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:19.939033  8736 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb4c86bcc704ac2add360ac3e5823df (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.943053  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb4c86bcc704ac2add360ac3e5823df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.939248  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 318340b5bf3b4640972d271299414f94 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.937870  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b476fe65810048fdb66dbeff393c8ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.945326  8940 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:19.948261  8940 ts_tablet_manager.cc:1403] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.155s	user 0.021s	sys 0.012s
I20260709 12:59:19.951399  8940 raft_consensus.cc:359] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.952138  8940 raft_consensus.cc:385] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.952440  8940 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.959566  8923 heartbeater.cc:499] Master 127.7.117.190:45111 was elected leader, sending a full tablet report...
I20260709 12:59:19.965505  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318340b5bf3b4640972d271299414f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.966506  8938 ts_tablet_manager.cc:1434] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.004s
I20260709 12:59:19.967597  8938 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:19.980432  8810 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb4c86bcc704ac2add360ac3e5823df (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.981719  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb4c86bcc704ac2add360ac3e5823df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.982137  8882 tablet_service.cc:1511] Processing CreateTablet for tablet 4bb4c86bcc704ac2add360ac3e5823df (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.986280  8738 tablet_service.cc:1511] Processing CreateTablet for tablet 3f192c6b771043a893d6fd420c9b9e7d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:19.987502  8738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f192c6b771043a893d6fd420c9b9e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.953222  8940 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:19.994066  8883 tablet_service.cc:1511] Processing CreateTablet for tablet 3f192c6b771043a893d6fd420c9b9e7d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:19.995234  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb4c86bcc704ac2add360ac3e5823df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.995869  8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f192c6b771043a893d6fd420c9b9e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.000345  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:20.001364  8879 tablet_service.cc:1511] Processing CreateTablet for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:20.001842  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.003532  8735 tablet_service.cc:1511] Processing CreateTablet for tablet cb3dbb5cac024c52a371cbe4f0f1e952 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:20.004719  8735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3dbb5cac024c52a371cbe4f0f1e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.005380  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.008502  8808 tablet_service.cc:1511] Processing CreateTablet for tablet cb3dbb5cac024c52a371cbe4f0f1e952 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:20.009764  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3dbb5cac024c52a371cbe4f0f1e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.011319  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 9402cffd3f284d28bf964bd56144a185 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:20.012656  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9402cffd3f284d28bf964bd56144a185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.033162  8948 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.033708  8948 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.040902  8939 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.041455  8939 ts_tablet_manager.cc:1403] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.141s	user 0.019s	sys 0.021s
I20260709 12:59:20.043272  8938 tablet_bootstrap.cc:654] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.044590  8939 raft_consensus.cc:359] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.051429  8939 raft_consensus.cc:385] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.057637  8940 ts_tablet_manager.cc:1434] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.109s	user 0.016s	sys 0.006s
I20260709 12:59:20.058531  8849 heartbeater.cc:499] Master 127.7.117.190:45111 was elected leader, sending a full tablet report...
I20260709 12:59:20.056072  8939 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.061874  8939 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.064327  8730 tablet_service.cc:1511] Processing CreateTablet for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:20.067292  8939 ts_tablet_manager.cc:1434] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:59:20.071789  8730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9662b1b1994a4a29b1ec6da7c3cd5f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.072264  8939 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.053138  8948 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.081562  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 3f192c6b771043a893d6fd420c9b9e7d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:20.083711  8881 tablet_service.cc:1511] Processing CreateTablet for tablet 9402cffd3f284d28bf964bd56144a185 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:20.084964  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9402cffd3f284d28bf964bd56144a185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.084722  8731 tablet_service.cc:1511] Processing CreateTablet for tablet d8087b240e9e4f5d9ea874980d0d7efa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:20.086366  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8087b240e9e4f5d9ea874980d0d7efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.089984  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f192c6b771043a893d6fd420c9b9e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.091972  8880 tablet_service.cc:1511] Processing CreateTablet for tablet cb3dbb5cac024c52a371cbe4f0f1e952 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:20.093593  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3dbb5cac024c52a371cbe4f0f1e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.099442  8945 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.100077  8945 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.106765  8945 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:20.116670  8938 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:20.138837  8736 tablet_service.cc:1511] Processing CreateTablet for tablet dada29d2380a453682f5dfef36dcf407 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:20.140249  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.160024  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada29d2380a453682f5dfef36dcf407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.148443  8940 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:20.172420  8732 tablet_service.cc:1511] Processing CreateTablet for tablet 932d7bc525ba4af99f7d6b52fc1d7337 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.173672  8737 tablet_service.cc:1511] Processing CreateTablet for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.174896  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.161387  8938 ts_tablet_manager.cc:1403] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.194s	user 0.014s	sys 0.002s
I20260709 12:59:20.179822  8938 raft_consensus.cc:359] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.180474  8938 raft_consensus.cc:385] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.180742  8938 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.181972  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932d7bc525ba4af99f7d6b52fc1d7337. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.168260  8787 leader_election.cc:343] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.186872  8939 tablet_bootstrap.cc:654] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.188800  8938 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.195022  8881 tablet_service.cc:1511] Processing CreateTablet for tablet d8087b240e9e4f5d9ea874980d0d7efa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:20.195077  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 90645da839314a85a4bdd72950c292b9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.196439  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8087b240e9e4f5d9ea874980d0d7efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.196784  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90645da839314a85a4bdd72950c292b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.196902  8938 ts_tablet_manager.cc:1434] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:59:20.198028  8938 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:20.206740  8879 tablet_service.cc:1511] Processing CreateTablet for tablet dada29d2380a453682f5dfef36dcf407 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:20.206920  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 932d7bc525ba4af99f7d6b52fc1d7337 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.208251  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932d7bc525ba4af99f7d6b52fc1d7337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.209002  8882 tablet_service.cc:1511] Processing CreateTablet for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.210285  8882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.220638  8940 tablet_bootstrap.cc:654] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.226472  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada29d2380a453682f5dfef36dcf407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.227898  8938 tablet_bootstrap.cc:654] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.227679  8883 tablet_service.cc:1511] Processing CreateTablet for tablet 90645da839314a85a4bdd72950c292b9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.236052  8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90645da839314a85a4bdd72950c292b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.236313  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 53b2ea39860b462cb7b87b739253a0c9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.243841  8809 tablet_service.cc:1511] Processing CreateTablet for tablet dada29d2380a453682f5dfef36dcf407 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:20.244207  8808 tablet_service.cc:1511] Processing CreateTablet for tablet 932d7bc525ba4af99f7d6b52fc1d7337 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.244737  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 90645da839314a85a4bdd72950c292b9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.245165  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada29d2380a453682f5dfef36dcf407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.245901  8810 tablet_service.cc:1511] Processing CreateTablet for tablet d8087b240e9e4f5d9ea874980d0d7efa (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:20.246306  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90645da839314a85a4bdd72950c292b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.246236  8813 tablet_service.cc:1511] Processing CreateTablet for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.243276  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 9402cffd3f284d28bf964bd56144a185 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:20.247277  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8087b240e9e4f5d9ea874980d0d7efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.251124  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932d7bc525ba4af99f7d6b52fc1d7337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.251751  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d3d159f1de4558ab5e2c0dcfc6f4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.264971  8811 tablet_service.cc:1511] Processing CreateTablet for tablet 53b2ea39860b462cb7b87b739253a0c9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.266286  8811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b2ea39860b462cb7b87b739253a0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.266013  8887 tablet_service.cc:1511] Processing CreateTablet for tablet 53b2ea39860b462cb7b87b739253a0c9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.267269  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b2ea39860b462cb7b87b739253a0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.274075  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.275377  8942 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.275868  8942 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.277493  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.278756  8942 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:20.280529  8789 leader_election.cc:343] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.291782  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9402cffd3f284d28bf964bd56144a185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.281082  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:20.299651  8939 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.300045  8947 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.300161  8939 ts_tablet_manager.cc:1403] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.228s	user 0.004s	sys 0.011s
I20260709 12:59:20.301604  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b2ea39860b462cb7b87b739253a0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.303486  8939 raft_consensus.cc:359] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.304203  8939 raft_consensus.cc:385] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.304484  8939 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.305703  8939 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.312310  8732 tablet_service.cc:1511] Processing CreateTablet for tablet 75401b4ca97a4c1eb75a016ff2124379 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.318893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.321437  8939 ts_tablet_manager.cc:1434] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
W20260709 12:59:20.322037  8864 leader_election.cc:343] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.322499  8939 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.322913  8736 tablet_service.cc:1511] Processing CreateTablet for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.318146  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75401b4ca97a4c1eb75a016ff2124379. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.327199  8863 leader_election.cc:343] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.327829  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 011f1e8a4b474eddb6b217393cbdd488 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.330164  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:20.329416  8881 tablet_service.cc:1511] Processing CreateTablet for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.331293  8945 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.332321  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011f1e8a4b474eddb6b217393cbdd488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.334089  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 410ca1c9d2fd42c68bdf6bb996508fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.338832  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 410ca1c9d2fd42c68bdf6bb996508fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.341351  8940 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:20.341827  8940 ts_tablet_manager.cc:1403] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.224s	user 0.011s	sys 0.002s
I20260709 12:59:20.348173  8940 raft_consensus.cc:359] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.351073  8940 raft_consensus.cc:385] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.351365  8940 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.352020  8940 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.354121  8940 ts_tablet_manager.cc:1434] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:59:20.354269  8812 tablet_service.cc:1511] Processing CreateTablet for tablet 75401b4ca97a4c1eb75a016ff2124379 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.355062  8940 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:20.355479  8812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75401b4ca97a4c1eb75a016ff2124379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.355635  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 75401b4ca97a4c1eb75a016ff2124379 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.356799  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75401b4ca97a4c1eb75a016ff2124379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.362257  8942 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.362777  8942 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.364826  8942 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.366566  8939 tablet_bootstrap.cc:654] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.370317  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.371533  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 410ca1c9d2fd42c68bdf6bb996508fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.376881  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.378188  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:20.379899  8715 leader_election.cc:343] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:20.380560  8715 leader_election.cc:343] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.364809  8940 tablet_bootstrap.cc:654] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.385771  8938 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:20.386212  8938 ts_tablet_manager.cc:1403] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.188s	user 0.021s	sys 0.017s
I20260709 12:59:20.389930  8938 raft_consensus.cc:359] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.390303  8808 tablet_service.cc:1511] Processing CreateTablet for tablet 029b57162d72422bba468c7d86b67fca (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.390656  8938 raft_consensus.cc:385] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.391089  8938 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.391557  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 029b57162d72422bba468c7d86b67fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.391916  8938 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.400512  8887 tablet_service.cc:1511] Processing CreateTablet for tablet 029b57162d72422bba468c7d86b67fca (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.401810  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 029b57162d72422bba468c7d86b67fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.406615  8938 ts_tablet_manager.cc:1434] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:59:20.407564  8938 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:20.408876  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 011f1e8a4b474eddb6b217393cbdd488 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.410532  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011f1e8a4b474eddb6b217393cbdd488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.411876  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 029b57162d72422bba468c7d86b67fca (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.413067  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 029b57162d72422bba468c7d86b67fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.417732  8732 tablet_service.cc:1511] Processing CreateTablet for tablet c782391e02e74664be7b5f81a3a9478d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:20.418114  8879 tablet_service.cc:1511] Processing CreateTablet for tablet 011f1e8a4b474eddb6b217393cbdd488 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.419248  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011f1e8a4b474eddb6b217393cbdd488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.428225  8807 tablet_service.cc:1511] Processing CreateTablet for tablet c782391e02e74664be7b5f81a3a9478d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:20.429520  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c782391e02e74664be7b5f81a3a9478d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.430634  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.432099  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.432834  8897 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
W20260709 12:59:20.434062  8712 leader_election.cc:343] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.434522  8712 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:20.435360  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c782391e02e74664be7b5f81a3a9478d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.435621  8942 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.436677  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:20.437606  8942 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.437983  8942 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.438300  8942 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.439363  8939 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.439818  8939 ts_tablet_manager.cc:1403] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.007s
I20260709 12:59:20.442651  8939 raft_consensus.cc:359] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.443332  8939 raft_consensus.cc:385] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.443608  8939 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.444330  8939 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.436193  8881 tablet_service.cc:1511] Processing CreateTablet for tablet c782391e02e74664be7b5f81a3a9478d (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:20.445768  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c782391e02e74664be7b5f81a3a9478d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.449345  8938 tablet_bootstrap.cc:654] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.454806  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 83a998af2b1141848fa02e188ffcd3c6 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.456008  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a998af2b1141848fa02e188ffcd3c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.446646  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 83a998af2b1141848fa02e188ffcd3c6 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.462069  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a998af2b1141848fa02e188ffcd3c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.448369  8942 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.462347  8939 ts_tablet_manager.cc:1434] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:59:20.467378  8939 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.463863  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb392189d54431c863a3b787fb95779 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.464725  8947 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.467890  8947 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.470090  8947 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.471633  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.473572  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.474747  8942 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.457181  8736 tablet_service.cc:1511] Processing CreateTablet for tablet 83a998af2b1141848fa02e188ffcd3c6 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.475656  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:20.476119  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a998af2b1141848fa02e188ffcd3c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.477542  8787 leader_election.cc:343] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:20.479089  8789 leader_election.cc:343] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.479486  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 12:59:20.481237  8715 leader_election.cc:343] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.482060  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:20.482568  8897 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.482930  8942 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.483508  8942 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.486196  8939 tablet_bootstrap.cc:654] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.492256  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.494364  8879 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb392189d54431c863a3b787fb95779 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.494706  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 234e3fa2a66f42f3a5a755f5f4adc858 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.494843  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.495620  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb392189d54431c863a3b787fb95779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.495800  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 234e3fa2a66f42f3a5a755f5f4adc858. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.496743  8712 leader_election.cc:343] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.497373  8947 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:20.500744  8715 leader_election.cc:343] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.501159  8715 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:20.502139  8938 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:20.502590  8938 ts_tablet_manager.cc:1403] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 12:59:20.502635  8965 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.502760  8808 tablet_service.cc:1511] Processing CreateTablet for tablet 234e3fa2a66f42f3a5a755f5f4adc858 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.504055  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 234e3fa2a66f42f3a5a755f5f4adc858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.511053  8737 tablet_service.cc:1511] Processing CreateTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.511504  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb392189d54431c863a3b787fb95779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.512320  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f573b86fda4b3e8fd982da9615dc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.491138  8942 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.511081  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 234e3fa2a66f42f3a5a755f5f4adc858 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.511093  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb392189d54431c863a3b787fb95779 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.518656  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 234e3fa2a66f42f3a5a755f5f4adc858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.519399  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb392189d54431c863a3b787fb95779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.519815  8945 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.520277  8945 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.523352  8945 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:20.523375  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.525112  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.525372  8938 raft_consensus.cc:359] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.526100  8938 raft_consensus.cc:385] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.526379  8938 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
W20260709 12:59:20.526623  8864 leader_election.cc:343] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.527074  8938 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:20.527486  8863 leader_election.cc:343] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.527942  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:20.530082  8939 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.530566  8939 ts_tablet_manager.cc:1403] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.005s
I20260709 12:59:20.531657  8897 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:20.532905  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:20.533457  8939 raft_consensus.cc:359] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.534170  8939 raft_consensus.cc:385] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.534420  8942 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.534492  8939 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.535291  8942 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:20.536031  8939 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.536171  8942 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.543797  8945 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.547281  8938 ts_tablet_manager.cc:1434] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 12:59:20.548418  8938 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:20.561391  8939 ts_tablet_manager.cc:1434] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:59:20.562600  8939 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.569604  8940 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:20.570176  8940 ts_tablet_manager.cc:1403] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.215s	user 0.018s	sys 0.017s
I20260709 12:59:20.569823  8807 tablet_service.cc:1511] Processing CreateTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.571074  8965 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.571456  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f573b86fda4b3e8fd982da9615dc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.571569  8965 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.586745  8645 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:20.600932  8965 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.601142  8879 tablet_service.cc:1511] Processing CreateTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.602427  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f573b86fda4b3e8fd982da9615dc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.602834  8938 tablet_bootstrap.cc:654] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.603335  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.603962  8823 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:20.604908  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.606011  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 12:59:20.607822  8712 leader_election.cc:343] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.607822  8810 tablet_service.cc:1511] Processing CreateTablet for tablet d3e6b55229c842abb4b6a8551191af12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.609853  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3e6b55229c842abb4b6a8551191af12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.612977  8885 tablet_service.cc:1511] Processing CreateTablet for tablet d3e6b55229c842abb4b6a8551191af12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.613476  8965 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.619609  8737 tablet_service.cc:1511] Processing CreateTablet for tablet e29b7d4482f943ce897e5a8d5bbb48c4 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.620208  8965 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.620642  8965 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.620795  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3e6b55229c842abb4b6a8551191af12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.623603  8939 tablet_bootstrap.cc:654] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.620805  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29b7d4482f943ce897e5a8d5bbb48c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.627336  8808 tablet_service.cc:1511] Processing CreateTablet for tablet e29b7d4482f943ce897e5a8d5bbb48c4 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.628640  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29b7d4482f943ce897e5a8d5bbb48c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.628536  8881 tablet_service.cc:1511] Processing CreateTablet for tablet e29b7d4482f943ce897e5a8d5bbb48c4 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.629886  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29b7d4482f943ce897e5a8d5bbb48c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.609076  8940 raft_consensus.cc:359] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.613963  8736 tablet_service.cc:1511] Processing CreateTablet for tablet d3e6b55229c842abb4b6a8551191af12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.630417  8940 raft_consensus.cc:385] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.637523  8940 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.638306  8940 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.640441  8940 ts_tablet_manager.cc:1434] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.000s
I20260709 12:59:20.648909  8940 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:20.652899  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3e6b55229c842abb4b6a8551191af12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.653825  8807 tablet_service.cc:1511] Processing CreateTablet for tablet f1a7b40e33c24527acba97c95cee9272 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:20.654933  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a7b40e33c24527acba97c95cee9272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.654856  8731 tablet_service.cc:1511] Processing CreateTablet for tablet f1a7b40e33c24527acba97c95cee9272 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:20.655998  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a7b40e33c24527acba97c95cee9272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.665603  8938 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:20.666113  8938 ts_tablet_manager.cc:1403] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.000s
I20260709 12:59:20.671533  8940 tablet_bootstrap.cc:654] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.682772  8965 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.689711  8945 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.690267  8945 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.692366  8945 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:20.692252  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:20.693037  8823 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.699028  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.699707  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:20.700516  8863 leader_election.cc:343] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.702054  8885 tablet_service.cc:1511] Processing CreateTablet for tablet f1a7b40e33c24527acba97c95cee9272 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:20.703415  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a7b40e33c24527acba97c95cee9272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.701508  8938 raft_consensus.cc:359] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.706672  8938 raft_consensus.cc:385] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.707082  8938 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.708022  8938 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.711475  8938 ts_tablet_manager.cc:1434] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
W20260709 12:59:20.712366  8864 leader_election.cc:343] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.712796  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:20.713871  8945 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.714247  8965 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.717811  8938 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:20.717942  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 12:59:20.719344  8712 leader_election.cc:343] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.720218  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 784f5ebc699a4812aaa4be775e13350a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:20.724274  8823 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:20.724159  8736 tablet_service.cc:1511] Processing CreateTablet for tablet b3894b893d1341819b9b2b487f3b2b26 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:20.731256  8939 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.725677  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 784f5ebc699a4812aaa4be775e13350a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:20.731940  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:20.729496  8881 tablet_service.cc:1511] Processing CreateTablet for tablet 784f5ebc699a4812aaa4be775e13350a (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:20.732720  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784f5ebc699a4812aaa4be775e13350a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.733141  8965 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.733013  8810 tablet_service.cc:1511] Processing CreateTablet for tablet b3894b893d1341819b9b2b487f3b2b26 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:20.731725  8939 ts_tablet_manager.cc:1403] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.169s	user 0.006s	sys 0.015s
I20260709 12:59:20.733898  8965 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:20.734269  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3894b893d1341819b9b2b487f3b2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.734949  8965 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.733433  8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784f5ebc699a4812aaa4be775e13350a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.731753  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3894b893d1341819b9b2b487f3b2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.740114  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784f5ebc699a4812aaa4be775e13350a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.736800  8939 raft_consensus.cc:359] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.753893  8939 raft_consensus.cc:385] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.754383  8939 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.756696  8939 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.760445  8939 ts_tablet_manager.cc:1434] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.026s	user 0.001s	sys 0.005s
I20260709 12:59:20.782545  8938 tablet_bootstrap.cc:654] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.784698  8879 tablet_service.cc:1511] Processing CreateTablet for tablet b3894b893d1341819b9b2b487f3b2b26 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:20.786005  8879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3894b893d1341819b9b2b487f3b2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.786347  8809 tablet_service.cc:1511] Processing CreateTablet for tablet eac82cfba1d244139d9b06d8a6c88635 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.787544  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac82cfba1d244139d9b06d8a6c88635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.789808  8887 tablet_service.cc:1511] Processing CreateTablet for tablet eac82cfba1d244139d9b06d8a6c88635 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.791018  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac82cfba1d244139d9b06d8a6c88635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.793589  8939 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.798105  8731 tablet_service.cc:1511] Processing CreateTablet for tablet fc45d3a2731b49daacb16548dc742815 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.798207  8808 tablet_service.cc:1511] Processing CreateTablet for tablet fc45d3a2731b49daacb16548dc742815 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.799264  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc45d3a2731b49daacb16548dc742815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.799463  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc45d3a2731b49daacb16548dc742815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.802441  8939 tablet_bootstrap.cc:654] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.785022  8736 tablet_service.cc:1511] Processing CreateTablet for tablet eac82cfba1d244139d9b06d8a6c88635 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.804925  8736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac82cfba1d244139d9b06d8a6c88635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.810166  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ad465a724ab581b9529345f73bba (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.810593  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ad465a724ab581b9529345f73bba (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.811466  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ad465a724ab581b9529345f73bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.811816  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ad465a724ab581b9529345f73bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.815532  8732 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ad465a724ab581b9529345f73bba (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.816783  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ad465a724ab581b9529345f73bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.810166  8885 tablet_service.cc:1511] Processing CreateTablet for tablet fc45d3a2731b49daacb16548dc742815 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.818586  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc45d3a2731b49daacb16548dc742815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.821385  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa55bf932444ccebb2b7f6583ead84f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.822582  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa55bf932444ccebb2b7f6583ead84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.826747  8965 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.827239  8965 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.830508  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.830956  8965 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.832502  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:20.833767  8940 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
W20260709 12:59:20.833904  8712 leader_election.cc:343] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.834518  8940 ts_tablet_manager.cc:1403] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.002s
W20260709 12:59:20.835023  8715 leader_election.cc:343] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.835485  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:20.837987  8965 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.820950  8645 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:20.838689  8883 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa55bf932444ccebb2b7f6583ead84f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.839082  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa55bf932444ccebb2b7f6583ead84f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.840000  8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa55bf932444ccebb2b7f6583ead84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.840313  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa55bf932444ccebb2b7f6583ead84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.837606  8940 raft_consensus.cc:359] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.849989  8940 raft_consensus.cc:385] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.850373  8940 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.851145  8940 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.853368  8940 ts_tablet_manager.cc:1434] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:59:20.870745  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 265e5f9784a6426482f5948fbfcc995c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.872745  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265e5f9784a6426482f5948fbfcc995c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.883291  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 265e5f9784a6426482f5948fbfcc995c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.888123  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265e5f9784a6426482f5948fbfcc995c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.900315  8732 tablet_service.cc:1511] Processing CreateTablet for tablet dc6dcf2342014ed6b65dd5af29d18d3b (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.901399  8940 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:20.901680  8732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6dcf2342014ed6b65dd5af29d18d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.903404  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 265e5f9784a6426482f5948fbfcc995c (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.904812  8965 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.905442  8965 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.907725  8965 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:20.910703  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.911803  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265e5f9784a6426482f5948fbfcc995c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.912331  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:20.912930  8715 leader_election.cc:343] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:20.913980  8712 leader_election.cc:343] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.914610  8939 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:20.915093  8945 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.915573  8945 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.917176  8887 tablet_service.cc:1511] Processing CreateTablet for tablet dc6dcf2342014ed6b65dd5af29d18d3b (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.918447  8887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6dcf2342014ed6b65dd5af29d18d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.918624  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:20.919261  8748 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:20.914446  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:20.920433  8863 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:20.921546  8965 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.921679  8945 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:20.922183  8945 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:20.922614  8945 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:20.923116  8945 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.926173  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:20.926234  8737 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac343934349411088cbf7a05932c2a5 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.926970  8823 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:20.927561  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac343934349411088cbf7a05932c2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.928907  8938 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:20.929955  8938 ts_tablet_manager.cc:1403] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.212s	user 0.016s	sys 0.043s
I20260709 12:59:20.915104  8939 ts_tablet_manager.cc:1403] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.154s	user 0.024s	sys 0.006s
I20260709 12:59:20.931109  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 7884996e8b84433f80124e36111f58ac (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.934340  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7884996e8b84433f80124e36111f58ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.922698  8940 tablet_bootstrap.cc:654] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.932873  8808 tablet_service.cc:1511] Processing CreateTablet for tablet dc6dcf2342014ed6b65dd5af29d18d3b (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.947842  8813 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac343934349411088cbf7a05932c2a5 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.949054  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac343934349411088cbf7a05932c2a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.949416  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:20.950337  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 7884996e8b84433f80124e36111f58ac (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.951543  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7884996e8b84433f80124e36111f58ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.954635  8945 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.951565  8938 raft_consensus.cc:359] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.957762  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:20.960453  8823 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.960477  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:20.961179  8748 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:20.961404  8883 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac343934349411088cbf7a05932c2a5 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.962688  8883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac343934349411088cbf7a05932c2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.966588  8939 raft_consensus.cc:359] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.966928  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 7884996e8b84433f80124e36111f58ac (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.967342  8939 raft_consensus.cc:385] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.967758  8939 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.967816  8748 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:20.968168  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7884996e8b84433f80124e36111f58ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.967860  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 3006e1c3fb5b4c66852cbc955265f1cb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:20.969054  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3006e1c3fb5b4c66852cbc955265f1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.968748  8939 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.969621  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6dcf2342014ed6b65dd5af29d18d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.970012  8863 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:20.971189  8939 ts_tablet_manager.cc:1434] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.005s
I20260709 12:59:20.974339  8939 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:20.958058  8938 raft_consensus.cc:385] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.974938  8938 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.975716  8938 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:20.973246  8823 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:20.986035  8978 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:20.989037  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 3006e1c3fb5b4c66852cbc955265f1cb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:20.990474  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3006e1c3fb5b4c66852cbc955265f1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.979761  8945 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.029870  8980 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.030519  8980 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.018359  8978 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:21.037493  8981 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.037976  8981 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.039144  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:21.043112  8938 ts_tablet_manager.cc:1434] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.113s	user 0.012s	sys 0.017s
I20260709 12:59:21.043514  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 3006e1c3fb5b4c66852cbc955265f1cb (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:21.044139  8938 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.044039  8980 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.044624  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3006e1c3fb5b4c66852cbc955265f1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.049854  8981 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.051402  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.052655  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.053047  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.053264  8748 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:21.054203  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:21.055174  8940 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.055335  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.055631  8940 ts_tablet_manager.cc:1403] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.169s	user 0.021s	sys 0.011s
I20260709 12:59:21.058630  8939 tablet_bootstrap.cc:654] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.059085  8807 tablet_service.cc:1511] Processing CreateTablet for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:21.062456  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.062304  8978 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.064939  8807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.075855  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:21.077244  8864 leader_election.cc:343] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.078930  8940 raft_consensus.cc:359] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.079613  8940 raft_consensus.cc:385] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.079875  8940 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.080528  8940 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:21.083750  8863 leader_election.cc:343] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.084223  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:21.085091  8983 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.085719  8808 tablet_service.cc:1511] Processing CreateTablet for tablet 48fca36d2cac4f99b1a750bcc17b5d12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 12:59:21.085974  8864 leader_election.cc:343] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.086293  8938 tablet_bootstrap.cc:654] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.086861  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fca36d2cac4f99b1a750bcc17b5d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.086604  8737 tablet_service.cc:1511] Processing CreateTablet for tablet a07b06e72d474091a0bcf7de181b509f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:21.087958  8737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a07b06e72d474091a0bcf7de181b509f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.094755  8733 tablet_service.cc:1511] Processing CreateTablet for tablet 48fca36d2cac4f99b1a750bcc17b5d12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:21.095796  8733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fca36d2cac4f99b1a750bcc17b5d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.096777  8863 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.099532  8813 tablet_service.cc:1511] Processing CreateTablet for tablet a07b06e72d474091a0bcf7de181b509f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:21.100518  8813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a07b06e72d474091a0bcf7de181b509f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.101480  8983 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.101821  8983 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.102110  8983 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.104573  8940 ts_tablet_manager.cc:1434] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 12:59:21.107226  8809 tablet_service.cc:1511] Processing CreateTablet for tablet 31c225a50048451db8964738c5db7540 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:21.108271  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c225a50048451db8964738c5db7540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.108729  8983 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.107299  8940 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.111454  8885 tablet_service.cc:1511] Processing CreateTablet for tablet a07b06e72d474091a0bcf7de181b509f (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:21.112514  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a07b06e72d474091a0bcf7de181b509f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.113631  8731 tablet_service.cc:1511] Processing CreateTablet for tablet 31c225a50048451db8964738c5db7540 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:21.122334  8880 tablet_service.cc:1511] Processing CreateTablet for tablet 48fca36d2cac4f99b1a750bcc17b5d12 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:21.114936  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c225a50048451db8964738c5db7540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.128607  8645 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.129611  8940 tablet_bootstrap.cc:654] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.131867  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fca36d2cac4f99b1a750bcc17b5d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.136715  8947 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.137233  8939 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.137167  8947 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.137722  8939 ts_tablet_manager.cc:1403] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.164s	user 0.016s	sys 0.024s
I20260709 12:59:21.139154  8947 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.140271  8939 raft_consensus.cc:359] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.140885  8939 raft_consensus.cc:385] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.141139  8939 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.141878  8939 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.142295  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.142901  8748 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.143335  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.143790  8939 ts_tablet_manager.cc:1434] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:21.144053  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:21.144714  8939 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.144850  8947 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.145215  8947 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.145552  8947 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:21.145977  8787 leader_election.cc:343] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.150226  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.150874  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.151429  8748 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:21.151551  8864 leader_election.cc:343] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.152705  8939 tablet_bootstrap.cc:654] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:21.161665  8715 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:21.164242  8885 tablet_service.cc:1511] Processing CreateTablet for tablet 31c225a50048451db8964738c5db7540 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:21.165776  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c225a50048451db8964738c5db7540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.170290  8940 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.170742  8940 ts_tablet_manager.cc:1403] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.001s
I20260709 12:59:21.174063  8940 raft_consensus.cc:359] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.174721  8940 raft_consensus.cc:385] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.175002  8940 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.175647  8940 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.152889  8983 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.180824  8965 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.181941  8748 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:21.183553  8863 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.185958  8978 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.186564  8978 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:21.187743  8978 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:21.189639  8712 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:21.190943  8947 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.195930  8731 tablet_service.cc:1511] Processing CreateTablet for tablet c6ea7c9c3095497eb93f5d5d412ad667 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:21.197492  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ea7c9c3095497eb93f5d5d412ad667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.202348  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.205687  8748 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.209448  8808 tablet_service.cc:1511] Processing CreateTablet for tablet c6ea7c9c3095497eb93f5d5d412ad667 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:21.210752  8808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ea7c9c3095497eb93f5d5d412ad667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.214612  8940 ts_tablet_manager.cc:1434] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.043s	user 0.014s	sys 0.015s
I20260709 12:59:21.220161  8809 tablet_service.cc:1511] Processing CreateTablet for tablet fd126387f2fb45f98d7ca0807566c688 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:21.221904  8809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd126387f2fb45f98d7ca0807566c688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.223424  8939 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.223963  8939 ts_tablet_manager.cc:1403] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.001s
I20260709 12:59:21.223388  8947 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.229743  8731 tablet_service.cc:1511] Processing CreateTablet for tablet fd126387f2fb45f98d7ca0807566c688 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:21.231200  8731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd126387f2fb45f98d7ca0807566c688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.234090  8939 raft_consensus.cc:359] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.235780  8940 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.235210  8939 raft_consensus.cc:385] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.238323  8939 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.239763  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 12:59:21.242058  8787 leader_election.cc:343] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.247191  8880 tablet_service.cc:1511] Processing CreateTablet for tablet fd126387f2fb45f98d7ca0807566c688 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:21.247339  8885 tablet_service.cc:1511] Processing CreateTablet for tablet c6ea7c9c3095497eb93f5d5d412ad667 (DEFAULT_TABLE table=test-workload [id=dec811e65e0b4bb6a470ee6b21001496]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:21.248405  8880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd126387f2fb45f98d7ca0807566c688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.246651  8965 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.249496  8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ea7c9c3095497eb93f5d5d412ad667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.249745  8965 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.251754  8965 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.254192  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.254822  8823 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:21.256393  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:21.257417  8939 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.262622  8938 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.263098  8938 ts_tablet_manager.cc:1403] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.219s	user 0.023s	sys 0.028s
I20260709 12:59:21.271865  8965 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.272284  8965 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.272665  8965 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.273799  8938 raft_consensus.cc:359] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.274547  8938 raft_consensus.cc:385] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.274821  8938 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.275521  8938 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.275887  8646 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.278714  8748 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.281495  8938 ts_tablet_manager.cc:1434] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:59:21.282451  8938 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.288488  8965 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.288486  8938 tablet_bootstrap.cc:654] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.288509  8940 tablet_bootstrap.cc:654] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.292047  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:21.293042  8947 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.293507  8965 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.293530  8947 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.294553  8947 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.295696  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.295848  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.296393  8823 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:21.297060  8712 leader_election.cc:343] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.297355  8978 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.304937  8965 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.305168  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.307166  8939 ts_tablet_manager.cc:1434] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.015s
I20260709 12:59:21.308605  8939 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.297796  8978 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.312525  8978 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
W20260709 12:59:21.313977  8712 leader_election.cc:343] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.315001  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.319706  8824 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:21.322690  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.324821  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:21.326614  8978 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.327080  8823 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:21.337576  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:21.339066  8965 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.339646  8965 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:21.340688  8965 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.345214  8996 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.345774  8996 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.348323  8996 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.346271  8938 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.349547  8938 ts_tablet_manager.cc:1403] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.022s
I20260709 12:59:21.349738  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.350065  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:21.351706  8712 leader_election.cc:343] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:21.352406  8715 leader_election.cc:343] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.352780  8938 raft_consensus.cc:359] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.352994  8715 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:21.353518  8938 raft_consensus.cc:385] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.353884  8938 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.354099  8996 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.354825  8938 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.360776  8938 ts_tablet_manager.cc:1434] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:21.361796  8938 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.368691  8978 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.368237  8645 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.374805  8939 tablet_bootstrap.cc:654] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.376142  8938 tablet_bootstrap.cc:654] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.378393  8940 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.378852  8940 ts_tablet_manager.cc:1403] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.143s	user 0.023s	sys 0.007s
I20260709 12:59:21.378454  8647 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.381000  8938 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.381461  8938 ts_tablet_manager.cc:1403] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.006s
I20260709 12:59:21.385849  8938 raft_consensus.cc:359] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.386497  8938 raft_consensus.cc:385] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.386751  8938 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.387285  8940 raft_consensus.cc:359] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.388216  8940 raft_consensus.cc:385] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.388513  8940 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.389190  8940 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.391225  8940 ts_tablet_manager.cc:1434] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:21.391983  8940 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.387377  8938 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.395602  8938 ts_tablet_manager.cc:1434] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:59:21.402245  8940 tablet_bootstrap.cc:654] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.407544  8938 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.407851  8939 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.408419  8939 ts_tablet_manager.cc:1403] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.100s	user 0.006s	sys 0.011s
I20260709 12:59:21.412312  8939 raft_consensus.cc:359] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.413174  8939 raft_consensus.cc:385] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.413473  8939 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.413592  8938 tablet_bootstrap.cc:654] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.414168  8939 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.417050  8939 ts_tablet_manager.cc:1434] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:21.418126  8939 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.424923  8939 tablet_bootstrap.cc:654] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.425450  8938 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.425994  8938 ts_tablet_manager.cc:1403] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:59:21.431182  8939 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.431483  8940 tablet_bootstrap.cc:492] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.431599  8939 ts_tablet_manager.cc:1403] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.002s
I20260709 12:59:21.431960  8940 ts_tablet_manager.cc:1403] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.008s
I20260709 12:59:21.433720  8939 raft_consensus.cc:359] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.434296  8939 raft_consensus.cc:385] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.434640  8939 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.434460  8940 raft_consensus.cc:359] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.435169  8940 raft_consensus.cc:385] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.435458  8940 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.435338  8939 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.436203  8938 raft_consensus.cc:359] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.436206  8940 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.436908  8938 raft_consensus.cc:385] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.437271  8939 ts_tablet_manager.cc:1434] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:21.437352  8938 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.438359  8938 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.438887  8940 ts_tablet_manager.cc:1434] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:59:21.439762  8939 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.440567  8940 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.440917  8947 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.441365  8947 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.443368  8947 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.443392  8938 ts_tablet_manager.cc:1434] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:59:21.444635  8938 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.445406  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.445765  8939 tablet_bootstrap.cc:654] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.445989  8898 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.446421  8940 tablet_bootstrap.cc:654] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.447073  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 12:59:21.448458  8789 leader_election.cc:343] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.453789  8938 tablet_bootstrap.cc:654] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.453630  8787 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:21.454491  8947 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.454967  8947 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.455392  8947 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.459777  8938 tablet_bootstrap.cc:492] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.460175  8938 ts_tablet_manager.cc:1403] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:59:21.462615  8938 raft_consensus.cc:359] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.462463  8947 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.463269  8938 raft_consensus.cc:385] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.463567  8938 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.464315  8938 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.466219  8938 ts_tablet_manager.cc:1434] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:21.466109  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 12:59:21.467690  8789 leader_election.cc:343] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.469053  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.469820  8898 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.474066  8938 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.479489  8898 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.480114  8938 tablet_bootstrap.cc:654] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.480960  8787 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:21.482292  8988 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.488072  8947 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.492105  8939 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.492496  8940 tablet_bootstrap.cc:492] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.492496  8939 ts_tablet_manager.cc:1403] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.008s
I20260709 12:59:21.493189  8940 ts_tablet_manager.cc:1403] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.029s
I20260709 12:59:21.493584  8947 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.494026  8947 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.495613  8939 raft_consensus.cc:359] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.496232  8939 raft_consensus.cc:385] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.496490  8939 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.496966  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.497161  8940 raft_consensus.cc:359] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.497089  8939 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.497609  8897 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.497957  8940 raft_consensus.cc:385] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.498287  8940 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.498889  8940 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.499359  8988 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.500083  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.499275  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.500273  8988 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.501085  8940 ts_tablet_manager.cc:1434] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:21.501052  8748 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.502130  8940 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.503852  8947 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.504395  8939 ts_tablet_manager.cc:1434] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:59:21.504560  8947 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.504942  8947 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.505262  8947 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.505704  8939 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.506350  8942 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.506788  8942 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.508411  8940 tablet_bootstrap.cc:654] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.508746  8942 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.510336  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.510566  8938 tablet_bootstrap.cc:492] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.511929  8939 tablet_bootstrap.cc:654] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.513325  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.514068  8898 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:21.515228  8712 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:21.516629  8942 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.522679  8939 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.523103  8939 ts_tablet_manager.cc:1403] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.007s
I20260709 12:59:21.523653  8947 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.516973  8938 ts_tablet_manager.cc:1403] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.008s
I20260709 12:59:21.520872  8940 tablet_bootstrap.cc:492] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.525621  8939 raft_consensus.cc:359] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.526021  8940 ts_tablet_manager.cc:1403] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.012s
I20260709 12:59:21.526270  8939 raft_consensus.cc:385] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.526702  8939 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.526621  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.527246  8897 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.527397  8939 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.528126  8938 raft_consensus.cc:359] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.528735  8938 raft_consensus.cc:385] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.529014  8940 raft_consensus.cc:359] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.529750  8940 raft_consensus.cc:385] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.529808  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.530145  8940 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.530382  8748 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.530966  8940 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.532406  8939 ts_tablet_manager.cc:1434] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:59:21.533334  8939 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.533469  8940 ts_tablet_manager.cc:1434] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:21.534111  8897 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.535071  8940 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.529013  8938 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.536430  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.536784  8938 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.537405  8947 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.538002  8988 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.538477  8988 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.539270  8988 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.540935  8646 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.548062  8938 ts_tablet_manager.cc:1434] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 12:59:21.549031  8938 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.549362  8942 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.550067  8748 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.559064  8978 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.559473  8938 tablet_bootstrap.cc:654] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.559547  8978 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.561725  8978 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.562417  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.560019  8940 tablet_bootstrap.cc:654] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.562925  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.563092  8748 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:21.562644  8939 tablet_bootstrap.cc:654] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:21.564189  8864 leader_election.cc:343] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.567634  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.569245  8978 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.569737  8978 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.570060  8978 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.572356  8938 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.572907  8938 ts_tablet_manager.cc:1403] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.006s
I20260709 12:59:21.565178  8647 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.576294  8938 raft_consensus.cc:359] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.577142  8938 raft_consensus.cc:385] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.581780  8938 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.582634  8938 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.584909  8988 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.585897  8988 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.588466  8988 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.589224  8940 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.589676  8940 ts_tablet_manager.cc:1403] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.013s
I20260709 12:59:21.590173  8939 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.590603  8939 ts_tablet_manager.cc:1403] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.008s
I20260709 12:59:21.592307  8939 raft_consensus.cc:359] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.592841  8939 raft_consensus.cc:385] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.593065  8939 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.585265  8938 ts_tablet_manager.cc:1434] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:21.594028  8824 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.592180  8940 raft_consensus.cc:359] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.594089  8939 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.594683  8940 raft_consensus.cc:385] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.595046  8940 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.595257  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.595868  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.594295  8938 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.595763  8940 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.596213  8939 ts_tablet_manager.cc:1434] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:21.597062  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.598227  8983 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.598683  8983 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.600303  8939 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.600592  8983 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.601651  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.601585  8978 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.602299  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.602597  8938 tablet_bootstrap.cc:654] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.603629  8978 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.604065  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.604667  8823 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:21.604961  8947 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.605331  8947 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.605638  8947 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.606665  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.607887  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.608489  8940 ts_tablet_manager.cc:1434] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
W20260709 12:59:21.608711  8864 leader_election.cc:343] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.609355  8940 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:21.609324  8978 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.609892  8978 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.610208  8978 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.601073  8983 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:59:21.617182  8939 tablet_bootstrap.cc:654] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.617127  8947 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.619843  8823 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.620440  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.620997  8749 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.622148  8988 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.623065  8988 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.624779  8988 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.625461  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.628844  8746 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.625559  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.631021  8748 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:21.635514  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:21.635944  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.636471  8748 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.638180  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.638785  8898 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:21.640223  9012 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.640616  9012 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.641743  9012 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.646211  8983 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.659981  8983 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.661130  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.666109  9012 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.653100  8749 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:21.674475  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.675086  8898 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.669916  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:21.676703  8947 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.678844  8947 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.679121  8897 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.679508  8947 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.680338  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.682761  8983 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.684064  8983 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:21.684850  8983 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.690146  8978 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.696266  8978 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.696592  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.698760  8821 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.701782  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.702347  8749 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.708463  9012 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.709731  8821 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:21.710830  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.711650  8978 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.712134  8978 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:21.680348  8947 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.712565  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.717598  8896 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.722009  8938 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:21.722465  8938 ts_tablet_manager.cc:1403] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.007s
I20260709 12:59:21.724036  8978 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.717608  8982 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:59:21.726521  8940 tablet_bootstrap.cc:654] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.727969  8938 raft_consensus.cc:359] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.781564  8938 raft_consensus.cc:385] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.781893  8938 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.738965  8749 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:21.731534  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.731372  8896 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.786718  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:21.787185  8938 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.792755  8938 ts_tablet_manager.cc:1434] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.070s	user 0.015s	sys 0.001s
I20260709 12:59:21.790304  8988 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.791706  8942 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.793764  8942 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.784368  8747 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.789758  8983 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.748574  8646 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.784125  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.799474  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.736589  9012 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 12:59:21.797199  8983 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.793757  8988 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.801124  8965 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:21.801472  8715 leader_election.cc:343] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.795753  8942 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:21.801767  8988 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.807490  8983 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.808139  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:21.808816  8749 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:21.811828  8938 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:21.815083  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:21.817521  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.818372  8983 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:21.818830  8983 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.819283  8983 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:21.809685  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:21.821327  8896 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:21.822419  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.823479  8942 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.823820  8942 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.825659  8942 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.831563  8983 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.832124  8942 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.832685  8898 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:59:21.836242  8747 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:21.836414  8646 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.837741  8939 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:21.840162  8939 ts_tablet_manager.cc:1403] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.240s	user 0.012s	sys 0.001s
I20260709 12:59:21.840754  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.841022  8898 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.844172  8983 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:21.843761  8939 raft_consensus.cc:359] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.845101  8939 raft_consensus.cc:385] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.845405  8939 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.846182  8939 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.846779  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:21.846848  9012 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.847391  8747 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:21.848079  8947 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:21.848517  8947 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:21.849639  8947 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.851020  8947 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:21.858532  8939 ts_tablet_manager.cc:1434] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:59:21.859849  8965 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.866858  8939 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:21.896559  8747 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:21.899055  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:21.901505  8978 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:21.902101  8938 tablet_bootstrap.cc:654] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.902736  8978 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:21.904132  8940 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:21.904560  8940 ts_tablet_manager.cc:1403] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.295s	user 0.006s	sys 0.010s
I20260709 12:59:21.907249  8940 raft_consensus.cc:359] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.907918  8940 raft_consensus.cc:385] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.908195  8940 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.909756  8978 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:21.910816  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:21.915465  8647 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.931527  8942 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:21.930802  8715 leader_election.cc:343] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.931417  8982 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.986424  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:21.987157  8897 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.988611  8988 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.983369  8982 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.006680  8988 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.009527  8982 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:22.009949  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:22.010756  8748 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:22.010804  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.012903  8863 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 12:59:22.015898  8864 leader_election.cc:343] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.016155  9026 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.016515  9026 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.016861  9026 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.067646  9026 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.068317  8942 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.075770  8942 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.077014  9026 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:22.080295  8939 tablet_bootstrap.cc:654] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.080101  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:22.080574  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:22.080843  8748 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:22.082247  8864 leader_election.cc:343] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.085001  8938 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
W20260709 12:59:22.071029  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:22.073338  8940 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.070441  8897 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.089267  8938 ts_tablet_manager.cc:1403] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.278s	user 0.009s	sys 0.013s
I20260709 12:59:22.095022  8940 ts_tablet_manager.cc:1434] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.190s	user 0.007s	sys 0.000s
I20260709 12:59:22.087181  8942 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.100380  8646 catalog_manager.cc:5697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.097818  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.122453  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.124207  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.129092  8940 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
W20260709 12:59:22.138031  8715 leader_election.cc:343] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.141784  8938 raft_consensus.cc:359] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.142480  8938 raft_consensus.cc:385] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.142743  8938 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.143391  9012 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.143476  8938 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.144586  8978 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.145820  8939 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.146273  8939 ts_tablet_manager.cc:1403] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.287s	user 0.014s	sys 0.050s
I20260709 12:59:22.139061  8748 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:22.153825  8940 tablet_bootstrap.cc:654] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.162793  8938 ts_tablet_manager.cc:1434] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.069s	user 0.003s	sys 0.003s
I20260709 12:59:22.163542  8938 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:22.166057  9041 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.166577  9041 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:22.166913  8646 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:22.167522  8896 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.168533  8863 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.170148  8988 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.171142  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.172178  8965 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.172537  8965 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.167384  9041 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.172865  8965 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.173353  8988 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.175889  9026 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.176374  9026 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:22.177162  9026 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.180208  8939 raft_consensus.cc:359] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.181057  8939 raft_consensus.cc:385] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.181399  8939 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.182265  8939 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.188465  8939 ts_tablet_manager.cc:1434] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 12:59:22.190979  8939 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.201135  8965 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.217510  8938 tablet_bootstrap.cc:654] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.221333  8824 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.221558  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:22.236272  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:22.237723  8898 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.242090  8939 tablet_bootstrap.cc:654] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:22.242614  8715 leader_election.cc:343] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.223316  8965 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.262466  8988 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.263160  9026 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.270121  9013 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.270663  9013 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.272559  8947 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.274766  8940 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.278421  8940 ts_tablet_manager.cc:1403] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.007s
I20260709 12:59:22.279604  9013 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.275027  9026 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.283140  9012 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.285197  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:22.283576  8898 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.294250  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.294492  8940 raft_consensus.cc:359] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.295439  8940 raft_consensus.cc:385] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.295794  8940 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.296679  8940 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.299677  8940 ts_tablet_manager.cc:1434] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 12:59:22.300874  8940 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.302098  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.323709  8938 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:22.324200  8938 ts_tablet_manager.cc:1403] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.161s	user 0.012s	sys 0.014s
I20260709 12:59:22.321498  8965 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.326073  8939 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.326558  8939 ts_tablet_manager.cc:1403] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.136s	user 0.009s	sys 0.019s
I20260709 12:59:22.333527  8939 raft_consensus.cc:359] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.334223  8939 raft_consensus.cc:385] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.334506  8939 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.335177  8939 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.340209  8647 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:22.336790  8939 ts_tablet_manager.cc:1434] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.000s
I20260709 12:59:22.346633  8939 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.325711  8965 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:22.347579  8747 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:22.347694  8965 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.350167  9033 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.357231  8787 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:22.361053  8938 raft_consensus.cc:359] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.361966  8938 raft_consensus.cc:385] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.362649  8938 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.395932  8940 tablet_bootstrap.cc:654] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.403728  9013 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:22.406144  9033 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.410426  8939 tablet_bootstrap.cc:654] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.414364  8938 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.420053  8938 ts_tablet_manager.cc:1434] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.096s	user 0.009s	sys 0.001s
I20260709 12:59:22.421001  8938 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:22.430801  8823 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:59:22.431228  8823 pending_rounds.cc:85] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 0
I20260709 12:59:22.431504  8938 tablet_bootstrap.cc:654] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.431563  8823 pending_rounds.cc:107] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:59:22.431955  8823 raft_consensus.cc:2889] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:59:22.435418  9033 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.455928  8940 tablet_bootstrap.cc:492] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.456418  8940 ts_tablet_manager.cc:1403] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.156s	user 0.031s	sys 0.008s
I20260709 12:59:22.459735  8940 raft_consensus.cc:359] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.460407  8940 raft_consensus.cc:385] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.460666  8940 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.461333  8940 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.463528  8940 ts_tablet_manager.cc:1434] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:59:22.464388  8940 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.464205  8646 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.466729  8646 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.470852  8940 tablet_bootstrap.cc:654] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.484181  8940 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.484994  8940 ts_tablet_manager.cc:1403] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 12:59:22.488965  8940 raft_consensus.cc:359] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.489889  8940 raft_consensus.cc:385] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.490234  8940 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.491278  8940 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.497082  8940 ts_tablet_manager.cc:1434] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:59:22.497512  8938 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:22.497984  8938 ts_tablet_manager.cc:1403] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.077s	user 0.041s	sys 0.013s
I20260709 12:59:22.497991  8940 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.498282  8939 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.498747  8939 ts_tablet_manager.cc:1403] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.152s	user 0.034s	sys 0.013s
I20260709 12:59:22.500761  8938 raft_consensus.cc:359] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.501204  8939 raft_consensus.cc:359] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.501590  8938 raft_consensus.cc:385] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.501856  8939 raft_consensus.cc:385] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.502007  8938 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.502177  8939 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.502998  8938 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.503453  8939 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.504581  8940 tablet_bootstrap.cc:654] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.505506  8939 ts_tablet_manager.cc:1434] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:22.506363  8939 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.511786  8939 tablet_bootstrap.cc:654] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.515950  8940 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.516387  8940 ts_tablet_manager.cc:1403] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:59:22.518651  8938 ts_tablet_manager.cc:1434] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:59:22.519134  8940 raft_consensus.cc:359] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.519834  8940 raft_consensus.cc:385] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.519909  8938 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:22.520156  8940 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.520999  8940 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.523252  8940 ts_tablet_manager.cc:1434] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:22.524158  8940 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.525081  8965 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.525573  8965 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.527704  8965 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.529572  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.530311  8940 tablet_bootstrap.cc:654] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.530222  8897 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.531489  8939 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.531759  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.532322  8823 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.530311  8938 tablet_bootstrap.cc:654] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.533499  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:22.539106  8965 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.539475  8965 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.539813  8965 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.542790  8940 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.543404  8940 ts_tablet_manager.cc:1403] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.004s
I20260709 12:59:22.544945  9044 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.545521  9044 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.546480  8940 raft_consensus.cc:359] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.547206  8940 raft_consensus.cc:385] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.548743  8940 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.549341  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.549501  8940 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.550386  8938 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
W20260709 12:59:22.550693  8715 leader_election.cc:343] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.550814  8938 ts_tablet_manager.cc:1403] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.008s
I20260709 12:59:22.552125  8940 ts_tablet_manager.cc:1434] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:59:22.552542  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.553066  8897 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.553592  8940 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.554263  8712 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.547640  9044 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.555476  8942 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.555855  8942 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.556207  8942 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.558013  8938 raft_consensus.cc:359] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.558669  8938 raft_consensus.cc:385] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.558933  8938 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.560191  8940 tablet_bootstrap.cc:654] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.560128  8938 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.567792  8938 ts_tablet_manager.cc:1434] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:59:22.568737  8938 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:22.569780  9044 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.579772  8965 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.580852  8938 tablet_bootstrap.cc:654] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.564711  8942 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.584816  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:22.531921  8939 ts_tablet_manager.cc:1403] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.012s
I20260709 12:59:22.585584  8823 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.589406  8942 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.589788  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:22.588596  8939 raft_consensus.cc:359] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.591342  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:22.591768  8939 raft_consensus.cc:385] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.592166  8939 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.593129  8939 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.592111  8898 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.595288  8939 ts_tablet_manager.cc:1434] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:22.596179  8939 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.597929  8822 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.601732  8939 tablet_bootstrap.cc:654] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:22.602110  8715 leader_election.cc:343] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.606179  8940 tablet_bootstrap.cc:492] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.606740  8940 ts_tablet_manager.cc:1403] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.053s	user 0.031s	sys 0.019s
I20260709 12:59:22.609694  8940 raft_consensus.cc:359] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.610321  8940 raft_consensus.cc:385] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.610574  8940 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.611114  8940 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.612993  8940 ts_tablet_manager.cc:1434] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:22.615237  8898 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.616504  8940 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.616345  8712 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:22.617450  9044 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.617908  9044 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:22.618695  9044 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.617926  8982 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:22.623634  8940 tablet_bootstrap.cc:654] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.660898  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:22.661726  8897 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.669960  8965 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.674044  8823 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.675638  8715 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:22.681633  8942 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.682616  9045 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.684696  8942 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:22.685956  8942 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.694891  8821 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.709719  9045 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.710240  9045 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.714129  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.714947  8896 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.716599  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:22.717607  9041 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.718055  9041 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.718480  9041 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.726869  8982 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.738674  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.739526  8820 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:22.753712  9045 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.766470  9041 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.770174  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:22.770949  8821 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.775211  8988 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.775861  8988 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.778507  8988 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.780478  8821 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.782398  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.783442  8898 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:22.785259  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:22.793231  8789 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:22.794752  8988 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.797938  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:22.818442  8645 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:22.829125  9076 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:59:22.833825  8897 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.847148  8645 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.851226  8965 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.851946  8965 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 12:59:22.853667  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55841.2us
I20260709 12:59:22.853022  8965 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.866441  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:22.867216  8897 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.869401  9041 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.870426  9012 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.871076  9012 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.874571  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:22.875427  8747 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:22.876298  9012 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.876569  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.877936  8988 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:22.878221  8787 leader_election.cc:343] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.879727  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:22.880800  9012 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.881160  9012 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.881525  9012 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.884974  8939 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.885588  8939 ts_tablet_manager.cc:1403] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.290s	user 0.049s	sys 0.046s
I20260709 12:59:22.887988  9012 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.888826  8939 raft_consensus.cc:359] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.889717  8939 raft_consensus.cc:385] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.889953  9012 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.890064  8939 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.890956  8939 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.891834  8942 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.893038  8942 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.893610  8939 ts_tablet_manager.cc:1434] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:22.894762  8939 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.902421  8939 tablet_bootstrap.cc:654] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.903695  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:22.904243  8747 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.903512  8644 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.908635  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 12:59:22.910157  8787 leader_election.cc:343] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.915333  8940 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.916013  8940 ts_tablet_manager.cc:1403] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.300s	user 0.140s	sys 0.137s
I20260709 12:59:22.919683  8940 raft_consensus.cc:359] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.921398  8940 raft_consensus.cc:385] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.921829  8940 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.922914  8940 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.925956  8940 ts_tablet_manager.cc:1434] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:22.927301  8940 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.935173  8940 tablet_bootstrap.cc:654] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.945500  8938 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:22.945930  8938 ts_tablet_manager.cc:1403] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.377s	user 0.107s	sys 0.141s
I20260709 12:59:22.946300  8940 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.946856  8940 ts_tablet_manager.cc:1403] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.005s
I20260709 12:59:22.948237  8897 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:22.948454  8938 raft_consensus.cc:359] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.949143  8938 raft_consensus.cc:385] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.949430  8938 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.949527  8940 raft_consensus.cc:359] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.950235  8940 raft_consensus.cc:385] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.950189  8938 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.950639  8940 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.951434  8940 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.951881  8747 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:22.952451  8938 ts_tablet_manager.cc:1434] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:22.954218  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:22.954932  8939 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.955343  8939 ts_tablet_manager.cc:1403] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.061s	user 0.032s	sys 0.015s
I20260709 12:59:22.955276  9012 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.956319  9012 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:22.956337  8940 ts_tablet_manager.cc:1434] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.001s
I20260709 12:59:22.957598  8940 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:22.957894  8939 raft_consensus.cc:359] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.957132  9012 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.958544  8939 raft_consensus.cc:385] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.958895  8939 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.959477  8939 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.961429  8939 ts_tablet_manager.cc:1434] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:22.962263  8939 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.965251  8940 tablet_bootstrap.cc:654] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.974201  8939 tablet_bootstrap.cc:654] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.976476  8938 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:22.981442  8965 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.981971  8965 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.982961  8940 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:22.983395  8940 ts_tablet_manager.cc:1403] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.014s
I20260709 12:59:22.983747  8939 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:22.984493  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:22.985659  8939 ts_tablet_manager.cc:1403] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.009s
I20260709 12:59:22.986047  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:22.986161  8940 raft_consensus.cc:359] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.986847  8940 raft_consensus.cc:385] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:22.987109  8715 leader_election.cc:343] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:22.988168  8712 leader_election.cc:343] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.988569  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:22.988658  8939 raft_consensus.cc:359] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.989267  8942 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.989503  8939 raft_consensus.cc:385] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.989920  8939 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.990044  8965 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:22.987129  8940 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.990767  8939 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.991770  8938 tablet_bootstrap.cc:654] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.991499  8940 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:22.993257  8939 ts_tablet_manager.cc:1434] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:22.998296  8939 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:22.998778  8940 ts_tablet_manager.cc:1434] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:59:22.999606  8938 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.000134  8938 ts_tablet_manager.cc:1403] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.007s
I20260709 12:59:22.999622  8940 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.003808  8939 tablet_bootstrap.cc:654] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.004026  8644 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.009230  8940 tablet_bootstrap.cc:654] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.009435  8938 raft_consensus.cc:359] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.010070  8938 raft_consensus.cc:385] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.010344  8938 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.010994  8938 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.013126  8938 ts_tablet_manager.cc:1434] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:59:23.013957  8938 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.027966  8938 tablet_bootstrap.cc:654] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.031869  8940 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.032334  8940 ts_tablet_manager.cc:1403] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.012s
I20260709 12:59:23.035007  8940 raft_consensus.cc:359] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.035444  8939 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.035667  8940 raft_consensus.cc:385] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.035876  8939 ts_tablet_manager.cc:1403] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.038s	user 0.001s	sys 0.024s
I20260709 12:59:23.035950  8940 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.036682  8940 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.038662  8940 ts_tablet_manager.cc:1434] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:59:23.038952  8939 raft_consensus.cc:359] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.039551  8939 raft_consensus.cc:385] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.039796  8940 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.039796  8939 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.042366  8938 tablet_bootstrap.cc:492] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.042164  8939 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.042882  8938 ts_tablet_manager.cc:1403] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 12:59:23.045871  8940 tablet_bootstrap.cc:654] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.045598  8938 raft_consensus.cc:359] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.046319  8938 raft_consensus.cc:385] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.046638  8938 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.047399  8938 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.048100  8939 ts_tablet_manager.cc:1434] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:59:23.049796  8939 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.049919  8938 ts_tablet_manager.cc:1434] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:23.052538  8938 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.053835  9012 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.054317  9012 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.056283  9012 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.057842  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.058422  8939 tablet_bootstrap.cc:654] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.058460  8747 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:23.058712  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.059916  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 12:59:23.060693  8787 leader_election.cc:343] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.060774  9012 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.061298  9012 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.061681  9012 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.068007  9012 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.073057  8988 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.073561  8988 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.075762  8988 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.076730  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:23.077142  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.077327  8747 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.077899  8896 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:23.082470  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.083454  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.083966  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 12:59:23.085042  8787 leader_election.cc:343] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.085616  9012 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.086153  9012 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.090484  8747 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:23.091178  8938 tablet_bootstrap.cc:654] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.091892  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:23.093113  8988 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.093578  8988 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:23.095156  8988 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.097003  9012 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.100401  8940 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.100966  8940 ts_tablet_manager.cc:1403] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.029s
I20260709 12:59:23.103664  8939 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.104179  8939 ts_tablet_manager.cc:1403] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.014s
I20260709 12:59:23.104206  8938 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.104837  8938 ts_tablet_manager.cc:1403] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.002s
I20260709 12:59:23.107745  8940 raft_consensus.cc:359] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.108253  8940 raft_consensus.cc:385] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.108464  8940 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.108700  8938 raft_consensus.cc:359] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.109031  8940 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.110635  8940 ts_tablet_manager.cc:1434] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 12:59:23.110688  8939 raft_consensus.cc:359] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.111351  8939 raft_consensus.cc:385] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.111387  8940 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.111709  8939 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.112363  8939 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.114226  8939 ts_tablet_manager.cc:1434] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:59:23.109354  8938 raft_consensus.cc:385] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.114555  8938 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.115198  8938 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.117136  8938 ts_tablet_manager.cc:1434] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 12:59:23.118238  8940 tablet_bootstrap.cc:654] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.120245  8939 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.119748  8645 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.125941  8938 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.135306  8939 tablet_bootstrap.cc:654] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:23.138896  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:23.157620  8965 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.159960  8940 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.160437  8940 ts_tablet_manager.cc:1403] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.003s
I20260709 12:59:23.160964  8938 tablet_bootstrap.cc:654] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.163318  8940 raft_consensus.cc:359] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.163985  8940 raft_consensus.cc:385] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.164244  8940 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.164832  8940 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.166795  8940 ts_tablet_manager.cc:1434] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:23.167171  8938 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.167579  8938 ts_tablet_manager.cc:1403] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.047s	user 0.007s	sys 0.005s
I20260709 12:59:23.167645  8940 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.170718  8938 raft_consensus.cc:359] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.171170  8939 tablet_bootstrap.cc:492] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.171355  8938 raft_consensus.cc:385] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.171583  8939 ts_tablet_manager.cc:1403] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.012s
I20260709 12:59:23.171602  8938 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.172384  8938 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.174114  8940 tablet_bootstrap.cc:654] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.174592  8938 ts_tablet_manager.cc:1434] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:59:23.175457  8938 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.178335  9044 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.178834  9044 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.180840  9044 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.182404  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.183012  8822 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.184410  8715 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.185235  9041 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.185611  9041 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.185914  9041 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.189590  9044 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:59:23.190532  8938 tablet_bootstrap.cc:654] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.191056  8939 raft_consensus.cc:359] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.191743  8939 raft_consensus.cc:385] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.192014  8939 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.192992  8939 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.194861  8939 ts_tablet_manager.cc:1434] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:59:23.195331  8982 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.195799  8982 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.199232  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.200376  8982 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 12:59:23.201443  8864 leader_election.cc:343] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.203095  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.203704  8747 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:23.204867  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:23.205338  8939 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.207115  9033 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.207281  8982 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.207567  9033 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.207970  9033 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.208014  8982 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.211598  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.212047  8939 tablet_bootstrap.cc:654] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.212381  8747 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:23.213968  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.214953  8983 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.214861  9033 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.215405  8983 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.215785  8983 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.217587  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:23.218147  8747 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.221989  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.222658  8823 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:23.223125  9044 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.223593  9044 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.225490  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.227150  8896 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.226647  8982 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:23.230602  8982 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.227356  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.222786  8983 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.233968  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:23.234501  8746 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.234584  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.235424  8822 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.242138  8746 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:23.244060  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.250245  9033 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:23.250758  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.251005  9033 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.251672  9033 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 12:59:23.252300  8864 leader_election.cc:343] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.253217  8822 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:23.228219  9044 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.254549  8747 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:23.226214  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.255343  8823 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.255950  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:23.257609  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.257932  8988 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.258417  9044 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.258855  9044 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.258526  8988 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.259218  9044 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.254698  8983 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:23.261441  8983 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.261973  8983 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:23.262368  8988 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.263748  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.263136  8983 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.265395  8940 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.265982  8940 ts_tablet_manager.cc:1403] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.010s
I20260709 12:59:23.227942  9041 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.269666  8982 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:23.270401  8787 leader_election.cc:343] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.269778  9044 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.271626  8982 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:23.271904  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.273638  9041 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:23.273603  8789 leader_election.cc:343] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.274161  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:23.274353  8897 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.275226  8988 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.275071  8940 raft_consensus.cc:359] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.275774  8940 raft_consensus.cc:385] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.276041  8940 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.276144  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.276717  8823 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.252696  9033 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.278106  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.282955  8940 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.292483  9041 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.294004  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.294585  8822 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.294732  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.294945  8823 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.295272  8897 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.282441  9044 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.297127  8940 ts_tablet_manager.cc:1434] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:59:23.298277  8940 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.314477  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.315127  8898 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.322286  8897 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.321081  8822 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
W20260709 12:59:23.321378  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65530us
I20260709 12:59:23.327085  8898 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.308590  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.327114  8645 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.329182  8645 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.327373  8712 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.331746  9041 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.332213  9041 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:23.333160  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.332991  9041 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.335927  8965 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.336398  8965 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:23.335768  8940 tablet_bootstrap.cc:654] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.337989  8942 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.343055  9044 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.337200  8965 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.345995  8939 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.346573  8939 ts_tablet_manager.cc:1403] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.141s	user 0.020s	sys 0.007s
I20260709 12:59:23.338121  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:23.350296  9108 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.381860  8938 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.382542  8938 ts_tablet_manager.cc:1403] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.207s	user 0.011s	sys 0.049s
W20260709 12:59:23.393026  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55722.8us
I20260709 12:59:23.423059  8939 raft_consensus.cc:359] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.425782  8939 raft_consensus.cc:385] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.426692  8939 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.427699  8939 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.426271  8938 raft_consensus.cc:359] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.429054  8938 raft_consensus.cc:385] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.429339  8938 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.427599  8644 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.441478  8988 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.464494  8938 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.468089  8939 ts_tablet_manager.cc:1434] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.000s
I20260709 12:59:23.471285  8983 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.471776  8983 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.473819  8983 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:23.478255  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.480659  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.481937  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:23.483072  9108 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.484892  8940 tablet_bootstrap.cc:492] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.485388  8940 ts_tablet_manager.cc:1403] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.187s	user 0.015s	sys 0.025s
I20260709 12:59:23.487174  8938 ts_tablet_manager.cc:1434] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.104s	user 0.015s	sys 0.009s
I20260709 12:59:23.488687  8938 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.487854  9044 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.493973  8939 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.494670  9041 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.497337  8940 raft_consensus.cc:359] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.498013  8940 raft_consensus.cc:385] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.498291  8940 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.498896  8940 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.500917  8940 ts_tablet_manager.cc:1434] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.000s
I20260709 12:59:23.501991  8940 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.494387  9044 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.507045  9044 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.516379  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.520423  8821 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.523646  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.530923  9113 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.525563  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.533274  9113 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.534289  9113 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.542690  8897 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.535563  8896 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.547894  8645 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.549990  9013 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.554780  9113 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.557788  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.558320  8898 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.558804  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.559381  8822 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.568337  9012 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.569648  8822 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.571633  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.572412  9045 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.572856  9045 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:23.573668  9045 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.575883  8939 tablet_bootstrap.cc:654] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.576865  9113 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.577841  8898 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.586807  8938 tablet_bootstrap.cc:654] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.595505  8940 tablet_bootstrap.cc:654] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:23.596014  8787 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:23.599148  8938 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.599745  8938 ts_tablet_manager.cc:1403] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.112s	user 0.004s	sys 0.007s
I20260709 12:59:23.602931  8938 raft_consensus.cc:359] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.603832  8938 raft_consensus.cc:385] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.604182  8938 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.605113  8938 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.604604  8646 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.607805  8938 ts_tablet_manager.cc:1434] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:59:23.608667  8938 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.616410  8938 tablet_bootstrap.cc:654] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.629171  8938 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.640223  8938 ts_tablet_manager.cc:1403] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.012s
I20260709 12:59:23.643805  8938 raft_consensus.cc:359] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.644948  8938 raft_consensus.cc:385] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.645624  8938 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.646546  8938 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.649606  8938 ts_tablet_manager.cc:1434] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 12:59:23.650866  8938 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.658566  8938 tablet_bootstrap.cc:654] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.671986  8939 tablet_bootstrap.cc:492] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.672493  8939 ts_tablet_manager.cc:1403] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.179s	user 0.036s	sys 0.044s
I20260709 12:59:23.674947  8939 raft_consensus.cc:359] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.677735  8939 raft_consensus.cc:385] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.678320  8940 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.678877  8940 ts_tablet_manager.cc:1403] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.177s	user 0.035s	sys 0.046s
I20260709 12:59:23.678375  8939 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.680387  8939 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.683547  8940 raft_consensus.cc:359] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.682426  8938 tablet_bootstrap.cc:492] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.684223  8940 raft_consensus.cc:385] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.684542  8940 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.684546  8938 ts_tablet_manager.cc:1403] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.006s
I20260709 12:59:23.685324  8940 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.687371  8940 ts_tablet_manager.cc:1434] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:23.687594  8938 raft_consensus.cc:359] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.688213  8938 raft_consensus.cc:385] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.688448  8938 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.688787  8940 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.689023  8938 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.691052  8938 ts_tablet_manager.cc:1434] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 12:59:23.691860  8938 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.684055  8939 ts_tablet_manager.cc:1434] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:59:23.693653  8939 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.696214  8940 tablet_bootstrap.cc:654] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.698845  8939 tablet_bootstrap.cc:654] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.702867  8938 tablet_bootstrap.cc:654] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.707949  8940 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.708352  8940 ts_tablet_manager.cc:1403] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.005s
I20260709 12:59:23.710630  8940 raft_consensus.cc:359] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.711253  8940 raft_consensus.cc:385] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.711515  8940 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.712246  8940 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.715160  8939 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.715585  8939 ts_tablet_manager.cc:1403] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:59:23.718035  8939 raft_consensus.cc:359] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.718645  8939 raft_consensus.cc:385] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.719007  8939 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.719640  8940 ts_tablet_manager.cc:1434] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:59:23.719719  8939 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.720358  9108 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.721544  8940 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.722443  8939 ts_tablet_manager.cc:1434] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:23.723337  8939 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.726838  8988 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.727303  8988 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.729231  8939 tablet_bootstrap.cc:654] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.729933  8938 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.729828  8988 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.730638  8938 ts_tablet_manager.cc:1403] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.012s
I20260709 12:59:23.731312  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.733434  8940 tablet_bootstrap.cc:654] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.733186  8938 raft_consensus.cc:359] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.733986  8938 raft_consensus.cc:385] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.734437  8938 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.735239  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.735183  8938 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.736675  8897 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:23.737963  8938 ts_tablet_manager.cc:1434] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:59:23.739153  8938 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.739342  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:23.740505  8988 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.741016  8988 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.741437  8988 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.744917  8940 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.745415  8940 ts_tablet_manager.cc:1403] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
W20260709 12:59:23.746412  8789 leader_election.cc:343] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.748764  8938 tablet_bootstrap.cc:654] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.748452  8940 raft_consensus.cc:359] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.749152  8940 raft_consensus.cc:385] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.749441  8940 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.750321  8940 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.752090  8940 ts_tablet_manager.cc:1434] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:23.753250  8940 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.757570  8988 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.759150  8940 tablet_bootstrap.cc:654] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.760432  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 12:59:23.761987  8789 leader_election.cc:343] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.764889  8938 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.765069  8940 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.765476  8938 ts_tablet_manager.cc:1403] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.009s
I20260709 12:59:23.768868  8938 raft_consensus.cc:359] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.769591  8938 raft_consensus.cc:385] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.769860  8988 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.769940  8938 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.770790  8938 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.773213  8938 ts_tablet_manager.cc:1434] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:59:23.765476  8940 ts_tablet_manager.cc:1403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 12:59:23.774080  8938 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.775234  8939 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.775707  8939 ts_tablet_manager.cc:1403] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.019s
I20260709 12:59:23.776137  8940 raft_consensus.cc:359] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.776813  8940 raft_consensus.cc:385] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.777103  8940 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.777791  8940 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.778363  8939 raft_consensus.cc:359] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.779109  8939 raft_consensus.cc:385] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.779429  8939 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.780151  8940 ts_tablet_manager.cc:1434] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:23.780337  8938 tablet_bootstrap.cc:654] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.780169  8939 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.781219  8940 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.782799  8939 ts_tablet_manager.cc:1434] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:59:23.784063  8939 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.784164  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.784966  8897 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.789338  8940 tablet_bootstrap.cc:654] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.790565  8939 tablet_bootstrap.cc:654] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.798445  8939 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.798939  8939 ts_tablet_manager.cc:1403] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:59:23.799841  9045 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.801676  8939 raft_consensus.cc:359] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.802383  8939 raft_consensus.cc:385] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.802685  8939 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.803440  8939 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.806414  8939 ts_tablet_manager.cc:1434] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:23.808861  8939 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.816558  8939 tablet_bootstrap.cc:654] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.807979  8940 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.817080  8940 ts_tablet_manager.cc:1403] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.000s
I20260709 12:59:23.820190  8940 raft_consensus.cc:359] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.821255  8940 raft_consensus.cc:385] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.821599  8940 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.822283  8940 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.824223  8940 ts_tablet_manager.cc:1434] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:59:23.827459  8940 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.828459  9108 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.833074  8940 tablet_bootstrap.cc:654] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.835772  9113 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.836473  8897 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:23.836313  9113 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.837563  8988 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.838105  8988 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.840516  8938 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.840448  9113 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.841111  8938 ts_tablet_manager.cc:1403] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.067s	user 0.018s	sys 0.047s
I20260709 12:59:23.843539  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:23.844434  8823 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.840510  8988 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.845804  9045 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.846056  9120 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.846683  9120 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:23.847666  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.847766  9120 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.851326  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.851976  8897 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.853593  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.854404  9113 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.854785  9113 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.855103  9113 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.856961  8983 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:23.858222  8787 leader_election.cc:343] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.867355  8939 tablet_bootstrap.cc:492] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.867815  8939 ts_tablet_manager.cc:1403] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.000s
I20260709 12:59:23.867749  8938 raft_consensus.cc:359] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.868870  8938 raft_consensus.cc:385] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.870559  8939 raft_consensus.cc:359] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.871189  8939 raft_consensus.cc:385] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.871412  8939 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.872392  8939 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.874845  8939 ts_tablet_manager.cc:1434] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 12:59:23.875653  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43587.7us
I20260709 12:59:23.877040  9045 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.877593  9045 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.878196  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.878786  8823 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:23.880045  9045 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.880285  9113 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.881850  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.882280  9113 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.882604  8897 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:23.883594  8939 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.884316  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.890334  8939 tablet_bootstrap.cc:654] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:23.893375  8789 leader_election.cc:343] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.894009  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:23.869197  8938 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.894690  8983 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.895303  8938 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.896793  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:23.898463  8820 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.901578  9012 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.902209  9012 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.902660  9012 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.903852  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.904407  8897 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.904712  9012 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.904913  8940 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.905449  8940 ts_tablet_manager.cc:1403] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.078s	user 0.029s	sys 0.016s
I20260709 12:59:23.908016  8940 raft_consensus.cc:359] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.908700  8940 raft_consensus.cc:385] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.909006  8940 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.909677  8940 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.912358  8940 ts_tablet_manager.cc:1434] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:59:23.897667  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:23.913998  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:23.913750  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:23.914649  8749 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:23.915167  8896 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:23.915827  8940 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.917940  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:23.919023  9120 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.919545  9120 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.919991  9120 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.923074  8715 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:23.923813  9045 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.927325  8940 tablet_bootstrap.cc:654] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.928380  9120 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.929834  8820 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.897774  8938 ts_tablet_manager.cc:1434] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 12:59:23.933049  8938 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:23.933676  9045 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.911293  8897 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:23.940099  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:23.941708  9045 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.942236  9045 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:23.942484  9113 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.943054  9045 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.943859  9120 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:23.944319  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:23.944898  8897 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.946993  8939 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:23.947517  8939 ts_tablet_manager.cc:1403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.009s
I20260709 12:59:23.949493  8938 tablet_bootstrap.cc:654] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.950011  8939 raft_consensus.cc:359] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.950762  8939 raft_consensus.cc:385] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.951064  8939 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.951846  8939 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.952201  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:23.952883  8749 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.961058  8940 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:23.978714  8940 ts_tablet_manager.cc:1403] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.005s
I20260709 12:59:23.975291  8644 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.984182  8939 ts_tablet_manager.cc:1434] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.000s
I20260709 12:59:23.985198  8939 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:23.981434  8940 raft_consensus.cc:359] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.986955  8940 raft_consensus.cc:385] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.987291  8940 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.993677  8940 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.974473  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:23.995615  8940 ts_tablet_manager.cc:1434] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 12:59:23.996500  8940 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:23.996717  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:23.988158  8938 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:23.997714  8988 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.993049  8939 tablet_bootstrap.cc:654] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.996799  8749 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:23.997979  8938 ts_tablet_manager.cc:1403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.008s
I20260709 12:59:23.998196  8988 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:23.999064  8988 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:23.987906  9113 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.001278  8938 raft_consensus.cc:359] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.006327  8938 raft_consensus.cc:385] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.006666  8938 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.013633  8938 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.016538  8938 ts_tablet_manager.cc:1434] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.000s
I20260709 12:59:24.017580  8938 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.032701  8965 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.034205  8983 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.034960  8939 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.034701  8983 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.035394  8939 ts_tablet_manager.cc:1403] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.016s
I20260709 12:59:24.036768  8983 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:24.037994  8939 raft_consensus.cc:359] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.038813  8939 raft_consensus.cc:385] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.039075  8939 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.039683  8939 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.041141  8939 ts_tablet_manager.cc:1434] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:59:24.002238  8940 tablet_bootstrap.cc:654] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.033371  8965 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.042232  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.042999  8823 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:24.044378  8864 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:24.046335  8983 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.046762  8983 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.047119  8983 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.046857  8644 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.048777  8938 tablet_bootstrap.cc:654] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.050515  8965 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.050917  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.051604  8823 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.052452  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.052999  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.053628  8749 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:24.055289  8715 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 12:59:24.056469  8712 leader_election.cc:343] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.057631  9044 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.058039  9044 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.058395  9044 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.058728  8983 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.060794  8983 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:24.061982  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:24.062542  8823 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.064509  8939 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.069278  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:24.069868  8747 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.064769  9044 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.072816  9044 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.073938  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.074343  8940 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.074802  8940 ts_tablet_manager.cc:1403] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.002s
I20260709 12:59:24.074717  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 12:59:24.075192  8712 leader_election.cc:343] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.075332  8820 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.076226  8747 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:24.077426  8940 raft_consensus.cc:359] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.078051  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.080247  8823 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:24.080368  9044 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.078073  8940 raft_consensus.cc:385] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.082098  8940 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.082333  8820 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.082872  8940 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.085040  8715 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:24.085363  8940 ts_tablet_manager.cc:1434] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:24.086166  9044 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.086401  8940 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.086638  9044 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:24.087525  8939 tablet_bootstrap.cc:654] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.087492  9044 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.089987  8983 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.090454  8983 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:24.091656  8983 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.100198  8938 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.100692  8938 ts_tablet_manager.cc:1403] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.083s	user 0.006s	sys 0.005s
I20260709 12:59:24.103191  8938 raft_consensus.cc:359] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.103825  8938 raft_consensus.cc:385] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.104070  8988 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.104580  8988 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.106590  8988 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.106027  8646 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.104115  8938 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.108423  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.108882  8938 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.108443  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 12:59:24.110916  8787 leader_election.cc:343] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.115386  8939 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
W20260709 12:59:24.115499  8789 leader_election.cc:343] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.116078  8939 ts_tablet_manager.cc:1403] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.007s
I20260709 12:59:24.115911  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:24.116971  8988 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.111095  8938 ts_tablet_manager.cc:1434] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:59:24.119684  9113 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.120337  8938 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.129910  8939 raft_consensus.cc:359] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.131930  8939 raft_consensus.cc:385] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.125679  8645 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.132249  8939 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.134464  8939 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.136659  8939 ts_tablet_manager.cc:1434] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.004s
I20260709 12:59:24.131053  8940 tablet_bootstrap.cc:654] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.137528  8939 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.140805  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:24.143658  8939 tablet_bootstrap.cc:654] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.147959  8982 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.148577  8982 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.152758  9108 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.153265  9108 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.154528  8897 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.155243  9108 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:24.156798  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.157449  8823 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:24.158654  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:24.159101  8939 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.159444  9108 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.159538  8939 ts_tablet_manager.cc:1403] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 12:59:24.159883  9108 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.160260  9108 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.162220  8939 raft_consensus.cc:359] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.162616  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.162922  8939 raft_consensus.cc:385] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.163203  8939 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.163229  8747 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:24.164537  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.164873  8939 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.165541  8982 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:24.166561  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.167645  8939 ts_tablet_manager.cc:1434] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:59:24.167948  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:24.168565  8939 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.169090  9045 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.169554  9045 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.188275  9108 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.190968  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.191164  9045 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:24.192471  8715 leader_election.cc:343] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.194813  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.196025  8938 tablet_bootstrap.cc:654] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.194865  8939 tablet_bootstrap.cc:654] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:24.197321  8712 leader_election.cc:343] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.197917  8712 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:24.198851  9045 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.208093  8939 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.208563  8939 ts_tablet_manager.cc:1403] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.002s
I20260709 12:59:24.212003  8939 raft_consensus.cc:359] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.213922  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:24.214246  8938 tablet_bootstrap.cc:492] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.214535  8749 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.214701  8938 ts_tablet_manager.cc:1403] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.095s	user 0.007s	sys 0.009s
I20260709 12:59:24.215826  9108 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:24.215993  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:24.216601  8820 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.213124  8939 raft_consensus.cc:385] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.217801  8939 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.218429  8982 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.220405  8940 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.220892  8940 ts_tablet_manager.cc:1403] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.135s	user 0.009s	sys 0.003s
I20260709 12:59:24.221688  8939 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.223945  8749 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:24.224352  8940 raft_consensus.cc:359] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.225095  8940 raft_consensus.cc:385] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.225170  8863 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.225455  8940 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.226253  8982 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.226212  8940 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.226790  8982 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:24.227622  8939 ts_tablet_manager.cc:1434] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.005s
I20260709 12:59:24.229064  8939 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.229509  8940 ts_tablet_manager.cc:1434] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:24.230406  8940 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.227586  8982 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:24.246112  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:24.246716  8938 raft_consensus.cc:359] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.247947  8938 raft_consensus.cc:385] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.248237  8938 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.248953  8938 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.247198  8820 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:24.251566  8940 tablet_bootstrap.cc:654] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.252619  8938 ts_tablet_manager.cc:1434] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:59:24.253670  8938 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.256201  9045 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.256829  9045 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.262921  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.263525  8820 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.264119  8940 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.264635  8940 ts_tablet_manager.cc:1403] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.005s
I20260709 12:59:24.264657  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.265954  9044 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.266211  8938 tablet_bootstrap.cc:654] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.266302  9044 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.266469  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.266741  9044 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.267617  8940 raft_consensus.cc:359] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:24.267843  8712 leader_election.cc:343] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.268222  9045 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.269418  8988 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.269577  8940 raft_consensus.cc:385] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.269856  8940 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.269896  8988 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.270552  8940 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.271842  8988 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.272665  8940 ts_tablet_manager.cc:1434] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.000s
I20260709 12:59:24.273267  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.273855  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:24.275391  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:24.275285  9044 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.276525  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.277134  8749 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:24.277997  8988 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.278400  8988 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.278754  8988 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.279982  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:24.280843  8939 tablet_bootstrap.cc:654] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.280731  8820 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.288412  8938 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.288211  8988 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.289033  8938 ts_tablet_manager.cc:1403] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 12:59:24.289793  8940 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.295457  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.296669  8940 tablet_bootstrap.cc:654] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.302305  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.303018  8898 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.303987  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:24.304736  8749 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.314077  8898 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:24.314913  8749 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:24.315286  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:24.292775  8938 raft_consensus.cc:359] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.315661  9044 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.322844  8988 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.320678  8820 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.323315  9120 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.323832  9120 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:24.324249  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.324646  9120 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.325083  9044 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.325788  9044 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:24.327220  9044 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.323328  8938 raft_consensus.cc:385] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.333138  8938 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.334267  9120 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:24.339442  8938 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.340788  8897 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.340837  8646 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:24.356544  8939 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.357101  8939 ts_tablet_manager.cc:1403] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.000s
I20260709 12:59:24.362087  8939 raft_consensus.cc:359] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.362846  8939 raft_consensus.cc:385] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.364115  8939 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.364494  8897 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.367012  8939 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.369391  8939 ts_tablet_manager.cc:1434] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:24.370357  8939 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.376907  8940 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.377436  8940 ts_tablet_manager.cc:1403] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.003s
I20260709 12:59:24.378070  8939 tablet_bootstrap.cc:654] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.380128  8940 raft_consensus.cc:359] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.380848  8940 raft_consensus.cc:385] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.381129  8940 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.381968  8940 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.384362  8940 ts_tablet_manager.cc:1434] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:24.385268  8940 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.397222  8940 tablet_bootstrap.cc:654] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.401358  8939 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.401804  8939 ts_tablet_manager.cc:1403] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.012s
I20260709 12:59:24.404377  8939 raft_consensus.cc:359] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.405040  8939 raft_consensus.cc:385] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.405364  8939 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.406068  8939 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.407564  8940 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.408066  8940 ts_tablet_manager.cc:1403] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.023s	user 0.001s	sys 0.012s
I20260709 12:59:24.408365  8939 ts_tablet_manager.cc:1434] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:24.408145  9013 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.409391  8939 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.411017  8940 raft_consensus.cc:359] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.411633  8940 raft_consensus.cc:385] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.411891  8940 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.412565  8940 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.414700  8940 ts_tablet_manager.cc:1434] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:59:24.415228  8939 tablet_bootstrap.cc:654] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.415570  8940 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.423606  8940 tablet_bootstrap.cc:654] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.427114  9013 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.427735  9013 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.431133  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.431842  8746 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:24.440510  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.441339  8988 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.441720  8988 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.442075  8988 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.445498  9013 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.451792  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:24.453135  8787 leader_election.cc:343] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.455391  8988 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.457751  8988 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.458927  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:24.459558  8746 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.460834  8938 ts_tablet_manager.cc:1434] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.171s	user 0.009s	sys 0.000s
I20260709 12:59:24.461112  8940 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.461581  8940 ts_tablet_manager.cc:1403] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.019s
I20260709 12:59:24.461623  8938 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.462862  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 12:59:24.463910  8787 leader_election.cc:343] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.464156  8940 raft_consensus.cc:359] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.464834  8940 raft_consensus.cc:385] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.465081  8940 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.465746  8940 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.466615  8939 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.467024  8939 ts_tablet_manager.cc:1403] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.015s
I20260709 12:59:24.468094  8940 ts_tablet_manager.cc:1434] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:24.468991  8940 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.469465  8939 raft_consensus.cc:359] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.470336  8939 raft_consensus.cc:385] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.470584  8939 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.471453  8939 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.474483  8938 tablet_bootstrap.cc:654] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.475361  8940 tablet_bootstrap.cc:654] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.475469  8939 ts_tablet_manager.cc:1434] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:24.476445  8939 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.478996  8746 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:24.484122  8939 tablet_bootstrap.cc:654] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.485066  8938 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.485520  8938 ts_tablet_manager.cc:1403] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 12:59:24.488132  8938 raft_consensus.cc:359] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.488809  8938 raft_consensus.cc:385] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.491055  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:24.493620  8988 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.489075  8938 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.494165  8988 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.494566  8938 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.496168  8988 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.496628  8988 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.497244  8988 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:24.498070  8988 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.499014  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.496620  8938 ts_tablet_manager.cc:1434] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
I20260709 12:59:24.502161  8938 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.519169  8982 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:24.523890  8939 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:24.524370  8939 ts_tablet_manager.cc:1403] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.009s
I20260709 12:59:24.543780  9185 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.544361  9185 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.547751  9185 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.548671  9185 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:24.548843  8787 leader_election.cc:343] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.549072  9185 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.551537  9185 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.598305  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.598897  8824 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.599500  8938 tablet_bootstrap.cc:654] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.599921  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.600564  9185 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.600962  9185 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.601380  9185 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.608062  9185 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.610066  9185 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.610893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:24.611380  9185 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.611826  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.612005  8824 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.612478  8892 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.614425  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.615002  8892 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.618101  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.618672  8892 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.619428  8824 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.620369  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.620988  9185 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.621416  9185 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:24.624346  9185 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.629648  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:24.633198  8715 leader_election.cc:343] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.634207  8938 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.634720  8938 ts_tablet_manager.cc:1403] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.133s	user 0.007s	sys 0.013s
I20260709 12:59:24.645816  8892 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.646780  8938 raft_consensus.cc:359] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.647531  8938 raft_consensus.cc:385] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.647815  8938 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.648432  8938 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.645820  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:24.654270  9185 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.654799  9185 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.655202  9185 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.660808  8938 ts_tablet_manager.cc:1434] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:59:24.661954  8938 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.662575  9185 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.665652  9185 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.666253  9185 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.667976  8938 tablet_bootstrap.cc:654] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.671335  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 12:59:24.672541  8715 leader_election.cc:343] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.672714  8892 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.674041  9185 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.674577  9185 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.673782  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.675791  8898 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.677493  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.678218  9185 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.678417  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:24.679930  8712 leader_election.cc:343] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.684024  9188 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.684819  8898 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:24.678167  8818 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.685904  9188 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.686379  9188 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.688326  9188 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.689417  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:24.690140  9185 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.690580  9185 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.691301  9185 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.692910  9108 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.693171  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.693768  8897 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.695312  8712 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:24.696110  9188 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.696462  9188 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.696807  9188 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.702334  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:24.703765  9013 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.704365  9013 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.707909  9188 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.708642  8940 tablet_bootstrap.cc:492] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:24.709209  8940 ts_tablet_manager.cc:1403] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.240s	user 0.017s	sys 0.016s
I20260709 12:59:24.716979  8939 raft_consensus.cc:359] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.717907  8939 raft_consensus.cc:385] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.718230  8939 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.719944  9196 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.720733  9196 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.730269  9194 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.732272  8940 raft_consensus.cc:359] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.736908  8940 raft_consensus.cc:385] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.735118  8939 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.737263  8940 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.711061  9185 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.741591  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:24.742488  8646 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.743732  9013 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.744463  9013 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.750136  8940 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.747437  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:24.843884  8939 ts_tablet_manager.cc:1434] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.319s	user 0.010s	sys 0.003s
I20260709 12:59:24.849056  8939 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:24.851250  8940 ts_tablet_manager.cc:1434] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.138s	user 0.002s	sys 0.005s
I20260709 12:59:24.853103  8940 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:24.850293  8748 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 12:59:24.846043  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104562us
I20260709 12:59:24.859494  9113 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.860064  9113 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:24.860216  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.848181  8822 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:24.860883  8898 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
W20260709 12:59:24.849036  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 128602us
W20260709 12:59:24.859488  9195 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:59:24.863451  9120 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.860857  9113 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.878615  9196 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.880491  8787 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:24.887030  8938 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:24.887650  8938 ts_tablet_manager.cc:1403] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.226s	user 0.010s	sys 0.029s
I20260709 12:59:24.895162  9185 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.891511  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.892750  9196 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.874789  9120 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.919915  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:24.926388  8939 tablet_bootstrap.cc:654] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.908367  9196 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:59:24.931428  8712 leader_election.cc:343] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.932005  9185 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.933425  9196 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.943754  8938 raft_consensus.cc:359] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.944824  8938 raft_consensus.cc:385] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.945204  8938 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.946125  8938 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:24.951433  9120 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.954355  8938 ts_tablet_manager.cc:1434] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:59:24.955545  8938 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:24.952669  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:24.959690  9188 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:24.952203  9199 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
W20260709 12:59:24.976089  8787 leader_election.cc:343] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.956459  9194 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:59:24.994515  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.014189  8938 tablet_bootstrap.cc:654] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.010810  8893 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.030053  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:25.039170  8940 tablet_bootstrap.cc:654] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:25.044781  9203 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:59:25.039866  9199 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:59:25.073076  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:25.086613  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.077881  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:25.102799  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:25.096470  8746 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:25.113567  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:25.116132  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.118886  9120 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.122031  9120 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.122479  9120 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.097858  8818 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.129555  8939 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:25.130214  8939 ts_tablet_manager.cc:1403] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.283s	user 0.019s	sys 0.007s
I20260709 12:59:25.139891  8938 tablet_bootstrap.cc:492] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:25.140381  8938 ts_tablet_manager.cc:1403] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.185s	user 0.010s	sys 0.011s
I20260709 12:59:25.145385  9188 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:25.131135  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 55793.5us
I20260709 12:59:25.124586  9196 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:25.250681  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 107260us
I20260709 12:59:25.268663  9202 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.269186  9202 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.271298  9202 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.287242  9190 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.287899  9190 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.284751  8819 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:25.364219  8939 raft_consensus.cc:359] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.364893  8939 raft_consensus.cc:385] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.365128  8939 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.293004  8893 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:25.373018  8939 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.380676  8939 ts_tablet_manager.cc:1434] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.250s	user 0.015s	sys 0.000s
W20260709 12:59:25.376897  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 233016us
I20260709 12:59:25.382982  8938 raft_consensus.cc:359] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:25.383471  9168 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.205s	user 0.004s	sys 0.025s
I20260709 12:59:25.383782  8938 raft_consensus.cc:385] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.384109  8938 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.384722  8938 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.376328  9196 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.389943  9198 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:59:25.392100  9190 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.397660  9120 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.407524  8938 ts_tablet_manager.cc:1434] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.267s	user 0.004s	sys 0.004s
I20260709 12:59:25.404845  8940 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:25.408209  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:25.409945  8940 ts_tablet_manager.cc:1403] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.557s	user 0.020s	sys 0.018s
W20260709 12:59:25.393920  8789 leader_election.cc:343] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.402962  8818 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:25.411667  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:25.411793  9013 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.414279  9197 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.410228  8747 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:25.410152  9120 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.435503  8940 raft_consensus.cc:359] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.438047  8940 raft_consensus.cc:385] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.438408  8940 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
W20260709 12:59:25.439201  9213 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.233s	user 0.007s	sys 0.004s
I20260709 12:59:25.439179  8940 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.441064  8939 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:25.441219  8940 ts_tablet_manager.cc:1434] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.001s
I20260709 12:59:25.442133  8940 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:25.448312  8647 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 65) took 1116 ms. Trace:
I20260709 12:59:25.448789  8647 rpcz_store.cc:276] 0709 12:59:24.332156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:24.336806 (+  4650us) service_pool.cc:224] Handling call
0709 12:59:25.448256 (+1111450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41915,"reported_tablets":11}
I20260709 12:59:25.449558  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:25.450369  8891 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:25.453662  8939 tablet_bootstrap.cc:654] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.454036  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.455142  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.457047  8896 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:25.457908  8898 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.459064  9045 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.459519  9045 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.461453  9045 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.467345  9120 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.470189  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:25.470841  8819 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:25.472384  8893 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.475651  8939 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:25.476153  8939 ts_tablet_manager.cc:1403] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.000s
I20260709 12:59:25.476831  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:25.477484  8819 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:25.479444  8938 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:25.479372  8939 raft_consensus.cc:359] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.480333  8939 raft_consensus.cc:385] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.480639  8939 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.481809  8939 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.484000  8939 ts_tablet_manager.cc:1434] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:59:25.484894  8939 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:25.485713  8938 tablet_bootstrap.cc:654] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.488834  8712 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.489769  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:25.491251  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:25.491978  8744 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:25.493645  8965 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.494104  8965 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:25.495391  8940 tablet_bootstrap.cc:654] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.502291  8938 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:25.502772  8938 ts_tablet_manager.cc:1403] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 12:59:25.503724  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:25.504343  8741 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.505532  8939 tablet_bootstrap.cc:654] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.494910  8965 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.507082  8938 raft_consensus.cc:359] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.515823  8938 raft_consensus.cc:385] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.516125  8938 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.516831  8938 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.519104  8938 ts_tablet_manager.cc:1434] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:59:25.519981  8938 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:25.526690  8741 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
W20260709 12:59:25.538401  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08001e8880 after lost signal to thread 7641
I20260709 12:59:25.545403  9044 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.545758  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:25.548627  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:25.555835  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 12:59:25.560796  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 83620.8us
I20260709 12:59:25.561164  9045 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.561674  9045 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:25.562677  9045 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.584033  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:25.584972  8892 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:25.594650  8938 tablet_bootstrap.cc:654] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.596843  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.597623  8965 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.598060  8965 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.598415  8965 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.598827  8940 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:25.599426  8940 ts_tablet_manager.cc:1403] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.157s	user 0.006s	sys 0.013s
W20260709 12:59:25.601722  8770 debug-util.cc:398] Leaking SignalData structure 0x7b080009dc00 after lost signal to thread 8681
W20260709 12:59:25.603050  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7c60 after lost signal to thread 8773
I20260709 12:59:25.603996  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.605160  9044 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.605537  9044 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.605449  8965 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.605963  9044 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.607573  9185 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.610265  8965 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.611739  9044 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.613255  9044 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.618021  9044 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.618578  9223 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:25.620879  9045 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.621534  8939 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:25.621984  8939 ts_tablet_manager.cc:1403] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.137s	user 0.038s	sys 0.061s
I20260709 12:59:25.623170  8940 raft_consensus.cc:359] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.623883  8940 raft_consensus.cc:385] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.624138  8940 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.624749  8940 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.624979  8939 raft_consensus.cc:359] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.625604  8939 raft_consensus.cc:385] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.625849  8939 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.626770  8940 ts_tablet_manager.cc:1434] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:59:25.626538  8939 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.627674  8940 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:25.628854  8939 ts_tablet_manager.cc:1434] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
W20260709 12:59:25.629559  8712 leader_election.cc:343] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.629699  8939 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:25.629117  8644 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:25.630849  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 12:59:25.604019  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08000ed9e0 after lost signal to thread 8848
I20260709 12:59:25.631619  9197 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.632234  9197 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:25.631999  8644 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:25.632701  8770 debug-util.cc:398] Leaking SignalData structure 0x7b080028d2c0 after lost signal to thread 8922
I20260709 12:59:25.633085  9197 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.636552  8939 tablet_bootstrap.cc:654] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.641583  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:25.641929  8819 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.651242  9120 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.653666  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:25.659349  8939 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:25.659861  8939 ts_tablet_manager.cc:1403] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.012s
I20260709 12:59:25.654568  9120 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:59:25.672012  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:25.696074  8938 tablet_bootstrap.cc:492] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:25.696574  8938 ts_tablet_manager.cc:1403] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.012s
I20260709 12:59:25.699029  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.699630  8893 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.707744  8939 raft_consensus.cc:359] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.708607  8939 raft_consensus.cc:385] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.708933  8939 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.709738  8939 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.711983  8939 ts_tablet_manager.cc:1434] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 12:59:25.697350  9044 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.712792  8938 raft_consensus.cc:359] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.715055  9044 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.713716  8938 raft_consensus.cc:385] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.719265  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.719875  8823 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.721141  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.721717  8822 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.700897  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.719269  8938 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.728546  8823 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:25.723841  8938 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.696574  9185 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:59:25.735441  8940 tablet_bootstrap.cc:654] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.738364  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.741389  8938 ts_tablet_manager.cc:1434] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:59:25.743369  8938 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:25.747582  8939 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:25.749814  9217 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:25.750988  8712 leader_election.cc:343] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.742345  9113 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.750332  9217 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.759749  9113 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:25.766023  9113 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.766089  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:25.777696  8816 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:25.786823  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:25.787489  8891 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:25.788591  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:25.787914  9044 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.789376  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.789171  8741 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:25.790124  9044 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.790581  9044 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.793121  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:25.793561  9044 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.793797  8821 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:25.794962  9217 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:25.798659  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:25.805259  9217 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.805687  9217 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.806033  9217 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.818678  8822 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:25.841229  8893 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:25.847483  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:25.850564  9201 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.851264  9201 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:25.851913  9185 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.852470  9185 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.852332  9201 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.864120  9185 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.867228  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:25.878109  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:25.880288  8712 leader_election.cc:343] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.916811  9044 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.918419  8938 tablet_bootstrap.cc:654] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:25.922768  8715 leader_election.cc:343] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.923214  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:25.924407  9045 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.926090  8939 tablet_bootstrap.cc:654] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.933008  8940 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:25.934456  8940 ts_tablet_manager.cc:1403] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.307s	user 0.015s	sys 0.019s
I20260709 12:59:25.935355  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:25.940047  8940 raft_consensus.cc:359] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.940773  8940 raft_consensus.cc:385] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.941118  8940 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.944837  8938 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:25.945087  9217 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.942010  9044 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:25.946141  8938 ts_tablet_manager.cc:1403] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.203s	user 0.013s	sys 0.009s
I20260709 12:59:25.951341  8940 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.957579  8647 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:25.960429  8647 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:25.969762  8940 ts_tablet_manager.cc:1434] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 12:59:25.971899  8940 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:25.969532  8896 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 1: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 12:59:25.973834  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:25.973244  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.975580  9217 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:25.975899  8820 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.977967  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:25.979643  8742 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:25.981578  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:25.983206  9108 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.986421  8939 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:25.986863  8939 ts_tablet_manager.cc:1403] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.239s	user 0.016s	sys 0.018s
I20260709 12:59:25.989459  8939 raft_consensus.cc:359] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.990101  8939 raft_consensus.cc:385] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.990365  8939 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.990993  8939 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:25.993768  8939 ts_tablet_manager.cc:1434] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:26.001327  8820 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:26.003774  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.005434  9190 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.006994  9190 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:26.007602  8939 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:26.001236  9185 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:59:26.007921  9190 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.014142  8644 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.016336  8644 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.017204  9185 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.017728  9185 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.018340  8644 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.020853  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:26.021787  8939 tablet_bootstrap.cc:654] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:26.022218  8712 leader_election.cc:343] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.023697  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:26.024865  8715 leader_election.cc:343] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.025261  8715 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.025919  9185 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.025944  9044 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.030789  8938 raft_consensus.cc:359] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.031476  8938 raft_consensus.cc:385] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.031739  8938 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.032371  8938 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.020277  8644 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.042096  8894 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.050241  8938 ts_tablet_manager.cc:1434] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.104s	user 0.004s	sys 0.003s
I20260709 12:59:26.051070  8938 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:26.054275  8988 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.058049  9190 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:59:26.072671  8938 tablet_bootstrap.cc:654] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.081573  9044 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:26.095723  8938 tablet_bootstrap.cc:492] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:26.097834  8938 ts_tablet_manager.cc:1403] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.006s
I20260709 12:59:26.096164  8940 tablet_bootstrap.cc:654] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.102881  9201 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.103757  8988 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.104200  8988 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.104386  8938 raft_consensus.cc:359] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.106132  8938 raft_consensus.cc:385] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.106387  8988 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.107462  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:26.107656  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:26.106789  8938 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.108074  8742 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
W20260709 12:59:26.108999  8787 leader_election.cc:343] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.109607  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.108880  8938 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.110635  9223 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.111001  9223 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.111317  9223 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.113844  8938 ts_tablet_manager.cc:1434] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:59:26.115125  8938 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:26.117802  8647 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.121953  9223 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.124208  8938 tablet_bootstrap.cc:654] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.128032  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:26.128908  8742 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.137035  9223 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.139609  8938 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:26.140066  8938 ts_tablet_manager.cc:1403] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.025s	user 0.001s	sys 0.008s
I20260709 12:59:26.140251  8742 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:26.141332  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:26.141968  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 12:59:26.142522  8787 leader_election.cc:343] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.143381  9223 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.144134  9223 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:26.143628  8644 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.145001  9223 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.145512  8939 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:26.145995  8939 ts_tablet_manager.cc:1403] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.027s
I20260709 12:59:26.148247  8938 raft_consensus.cc:359] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.148911  8938 raft_consensus.cc:385] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.149160  8938 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.149755  8938 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.151208  8939 raft_consensus.cc:359] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.151816  8939 raft_consensus.cc:385] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.152087  8939 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.152683  8939 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.156472  8939 ts_tablet_manager.cc:1434] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:26.157441  8939 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:26.161381  8938 ts_tablet_manager.cc:1434] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:59:26.162300  8938 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:26.164180  9201 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.164644  9201 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.164911  9190 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.167936  9201 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.173645  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:26.175351  8712 leader_election.cc:343] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.168527  8939 tablet_bootstrap.cc:654] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.169867  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:26.175602  8938 tablet_bootstrap.cc:654] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.179821  8940 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:26.180382  8940 ts_tablet_manager.cc:1403] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.209s	user 0.011s	sys 0.001s
W20260709 12:59:26.183339  8715 leader_election.cc:343] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.183802  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.184607  9201 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.190598  8940 raft_consensus.cc:359] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.191350  8940 raft_consensus.cc:385] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.191692  8940 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.193130  8940 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.194664  8940 ts_tablet_manager.cc:1434] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:59:26.195793  8940 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:26.200927  8939 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:26.246734  8939 ts_tablet_manager.cc:1403] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.089s	user 0.011s	sys 0.002s
I20260709 12:59:26.257697  9223 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:26.268774  9044 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:59:26.295269  8940 tablet_bootstrap.cc:654] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.297725  8939 raft_consensus.cc:359] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.298746  8939 raft_consensus.cc:385] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.299067  8939 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.300129  8939 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.304226  8938 tablet_bootstrap.cc:492] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:26.305092  8938 ts_tablet_manager.cc:1403] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.143s	user 0.015s	sys 0.000s
I20260709 12:59:26.313921  8939 ts_tablet_manager.cc:1434] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.006s
I20260709 12:59:26.315650  8938 raft_consensus.cc:359] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.315339  8647 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.316370  8938 raft_consensus.cc:385] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.316648  8938 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.317348  8938 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.322597  8938 ts_tablet_manager.cc:1434] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:59:26.324359  8938 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:26.326315  8939 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:26.337471  8940 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:26.339727  8940 ts_tablet_manager.cc:1403] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.144s	user 0.025s	sys 0.014s
I20260709 12:59:26.337641  8939 tablet_bootstrap.cc:654] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.340577  8938 tablet_bootstrap.cc:654] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.348438  9190 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.348975  9190 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.362466  9217 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.363165  9217 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:26.446779  9270 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:59:26.487748  9205 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.488538  9205 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.491974  9223 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.492591  9223 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.510829  9205 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:26.511186  8940 raft_consensus.cc:359] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.512166  8940 raft_consensus.cc:385] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.512456  8940 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.515528  8940 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.523138  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:26.524804  9217 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:26.543968  8940 ts_tablet_manager.cc:1434] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.204s	user 0.005s	sys 0.004s
I20260709 12:59:26.554823  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:26.555786  8749 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:26.565665  9190 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.568481  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:26.551086  9223 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.569521  8820 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:26.573661  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:26.574491  8940 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:26.575387  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:26.574528  8744 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:26.584774  9108 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.585391  9108 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.586494  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:26.587560  9108 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:26.598367  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:26.599627  8787 leader_election.cc:343] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.600777  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:26.601446  8820 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:26.610644  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:26.611553  9271 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.611968  9271 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.612308  9271 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:26.613211  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71964.8us
I20260709 12:59:26.614284  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:26.615412  9272 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.616142  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:26.618175  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.620333  9012 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.620761  9012 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.621182  9012 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.622776  9272 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.624413  9271 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.635373  9108 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.648413  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:26.651185  8712 leader_election.cc:343] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.652042  8940 tablet_bootstrap.cc:654] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.639123  9185 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:59:26.635522  9190 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 12:59:26.671125  8715 leader_election.cc:343] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.671742  8715 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.672381  9012 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.676239  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:26.676892  8824 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.681847  9200 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.688359  8939 tablet_bootstrap.cc:492] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:26.688932  8939 ts_tablet_manager.cc:1403] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.363s	user 0.022s	sys 0.038s
I20260709 12:59:26.666493  9271 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:26.708010  9012 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.710915  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 12:59:26.713061  8787 leader_election.cc:343] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.716437  8939 raft_consensus.cc:359] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.718394  8938 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:26.719003  8938 ts_tablet_manager.cc:1403] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.395s	user 0.023s	sys 0.043s
I20260709 12:59:26.719043  9108 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.725163  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:26.728554  9012 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.731827  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:26.733155  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:26.733980  9108 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.734369  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:26.734895  9108 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:26.717146  8939 raft_consensus.cc:385] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.736522  8939 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.736186  9108 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.738512  8938 raft_consensus.cc:359] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.740046  8938 raft_consensus.cc:385] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.744617  8938 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.737320  8939 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:26.746028  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 94165.6us
I20260709 12:59:26.747812  8744 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:26.758968  8939 ts_tablet_manager.cc:1434] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.000s
I20260709 12:59:26.754783  8938 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.770339  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:26.771003  8744 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.776142  9188 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.776628  9188 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.778646  9188 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.781592  8938 ts_tablet_manager.cc:1434] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.001s
I20260709 12:59:26.766884  9190 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:59:26.784380  8744 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:26.787945  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:26.788441  9274 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:59:26.791935  8939 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
W20260709 12:59:26.794015  8715 leader_election.cc:343] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.792775  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:26.764655  9012 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 12:59:26.794759  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.796195  8940 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:26.797138  8940 ts_tablet_manager.cc:1403] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.223s	user 0.028s	sys 0.027s
I20260709 12:59:26.800979  9197 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:26.807842  8712 leader_election.cc:343] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.809232  8938 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:26.808292  8712 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:26.815707  9201 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.813301  9197 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 12:59:26.854488  9282 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:59:26.904924  8940 raft_consensus.cc:359] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.905782  8940 raft_consensus.cc:385] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.906153  8940 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.913568  8940 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.915426  9197 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.936115  8938 tablet_bootstrap.cc:654] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.956303  9199 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.958128  9199 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:26.960400  9199 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:26.966284  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:26.972777  8939 tablet_bootstrap.cc:654] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:26.993753  8787 leader_election.cc:343] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.003588  8940 ts_tablet_manager.cc:1434] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.206s	user 0.020s	sys 0.005s
I20260709 12:59:27.014418  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.015184  8742 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.016582  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 12:59:27.018784  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:27.025635  9223 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.026111  9223 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.026459  9223 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.031562  8644 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:27.036249  8940 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.041697  9223 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.051870  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:27.052523  8742 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.061995  8939 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.062343  8938 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.062448  8939 ts_tablet_manager.cc:1403] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.296s	user 0.012s	sys 0.006s
I20260709 12:59:27.062757  8938 ts_tablet_manager.cc:1403] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.255s	user 0.008s	sys 0.031s
I20260709 12:59:27.069989  8742 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.072487  8939 raft_consensus.cc:359] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.077476  9223 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.081588  8939 raft_consensus.cc:385] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.081899  8939 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.083179  8939 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.084988  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.085397  8938 raft_consensus.cc:359] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.086078  8938 raft_consensus.cc:385] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.086338  8938 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.087041  8938 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.088631  8938 ts_tablet_manager.cc:1434] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:59:27.089949  8938 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.092244  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.092890  8939 ts_tablet_manager.cc:1434] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 12:59:27.093096  9223 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.093720  9223 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:27.093995  8939 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.094630  9223 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:27.102638  8787 leader_election.cc:343] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.103580  8938 tablet_bootstrap.cc:654] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.124017  9108 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.124622  8940 tablet_bootstrap.cc:654] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.124547  9108 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.126736  9108 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:27.129321  8939 tablet_bootstrap.cc:654] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.129361  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.130199  8742 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:27.131175  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:27.133939  8864 leader_election.cc:343] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.134696  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:27.135982  9108 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.136339  9108 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.136664  9108 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.146967  9108 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.149605  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:27.150223  8742 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.151126  8938 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.150833  9108 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:27.151571  8938 ts_tablet_manager.cc:1403] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.001s
I20260709 12:59:27.153409  8645 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.154417  8938 raft_consensus.cc:359] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.155071  8938 raft_consensus.cc:385] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.155316  8938 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.156087  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:27.155918  8938 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 12:59:27.157586  8864 leader_election.cc:343] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.158105  8938 ts_tablet_manager.cc:1434] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:59:27.159071  8938 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.161124  8742 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:27.162437  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 12:59:27.163579  8939 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.164047  8939 ts_tablet_manager.cc:1403] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.006s
I20260709 12:59:27.166180  8939 raft_consensus.cc:359] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.167924  8939 raft_consensus.cc:385] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.168184  8939 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.165083  9108 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.169029  8939 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.169539  9108 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:27.170395  9108 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.171015  8939 ts_tablet_manager.cc:1434] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:27.172206  8939 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.175278  8940 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.175745  8940 ts_tablet_manager.cc:1403] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.156s	user 0.012s	sys 0.005s
I20260709 12:59:27.178592  8940 raft_consensus.cc:359] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.179211  8940 raft_consensus.cc:385] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.179471  8940 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.179800  8939 tablet_bootstrap.cc:654] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.183712  8938 tablet_bootstrap.cc:654] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.188227  8939 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.198894  8940 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.206771  8940 ts_tablet_manager.cc:1434] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:59:27.200384  8645 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.204679  9190 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.207695  8940 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.207463  9190 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.209342  9190 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.210335  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.210335  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:27.212013  8712 leader_election.cc:343] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:27.212450  8715 leader_election.cc:343] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.212826  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:27.213766  9190 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.214850  8939 ts_tablet_manager.cc:1403] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.003s
I20260709 12:59:27.216686  8938 tablet_bootstrap.cc:492] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.217105  8938 ts_tablet_manager.cc:1403] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:59:27.217826  8939 raft_consensus.cc:359] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.218554  8939 raft_consensus.cc:385] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.218822  8939 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.221165  8940 tablet_bootstrap.cc:654] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.221985  8646 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:27.222735  8742 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.219458  8939 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.229884  8939 ts_tablet_manager.cc:1434] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:59:27.230949  8939 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.231724  9188 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.232206  9188 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.235314  8938 raft_consensus.cc:359] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.237005  8938 raft_consensus.cc:385] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.237411  8938 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.242806  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.246021  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 12:59:27.251801  8712 leader_election.cc:343] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:27.255956  8715 leader_election.cc:343] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.256475  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:27.234376  9188 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.257860  8938 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.259718  9190 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.259930  8938 ts_tablet_manager.cc:1434] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.000s
I20260709 12:59:27.260620  9108 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.261243  8938 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.262748  8939 tablet_bootstrap.cc:654] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.283136  9108 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:59:27.295605  8940 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.296051  8940 ts_tablet_manager.cc:1403] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.030s
I20260709 12:59:27.299896  9108 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.300393  9108 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.301843  8940 raft_consensus.cc:359] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.302471  8940 raft_consensus.cc:385] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.302722  8940 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.303534  9108 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:27.303316  8940 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.303452  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.303396  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.305841  8940 ts_tablet_manager.cc:1434] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:59:27.306185  8744 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:27.306733  8940 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.312039  8863 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:27.314051  9272 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.322419  8939 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.322747  8938 tablet_bootstrap.cc:654] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.322903  8939 ts_tablet_manager.cc:1403] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.005s
I20260709 12:59:27.327402  8939 raft_consensus.cc:359] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.328051  8939 raft_consensus.cc:385] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.328308  8939 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.329097  8940 tablet_bootstrap.cc:654] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.328922  8939 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.342099  8939 ts_tablet_manager.cc:1434] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:59:27.342998  8939 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.356475  9272 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.370024  8938 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.370543  8938 ts_tablet_manager.cc:1403] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.110s	user 0.011s	sys 0.021s
I20260709 12:59:27.370024  8939 tablet_bootstrap.cc:654] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.373934  8940 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.374577  8938 raft_consensus.cc:359] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.375535  8938 raft_consensus.cc:385] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.375955  8938 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.376794  8938 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.379294  8938 ts_tablet_manager.cc:1434] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:59:27.381345  8938 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.380513  9272 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.383847  9272 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.385700  9272 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:27.386230  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.386976  8744 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 1: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 12:59:27.387784  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.389194  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:27.390156  8938 tablet_bootstrap.cc:654] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.390228  9272 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.374378  8940 ts_tablet_manager.cc:1403] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.008s
I20260709 12:59:27.396402  8939 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.396389  8940 raft_consensus.cc:359] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.396952  8939 ts_tablet_manager.cc:1403] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.000s
I20260709 12:59:27.397147  8940 raft_consensus.cc:385] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.397473  8940 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.398110  8940 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.399422  8939 raft_consensus.cc:359] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.400004  8939 raft_consensus.cc:385] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.400137  8940 ts_tablet_manager.cc:1434] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:27.400250  8939 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.400931  8939 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.401437  8940 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.402926  8939 ts_tablet_manager.cc:1434] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:27.406405  9272 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.410104  8939 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.413861  8940 tablet_bootstrap.cc:654] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.419690  8938 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.420384  8938 ts_tablet_manager.cc:1403] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.005s
I20260709 12:59:27.422395  8939 tablet_bootstrap.cc:654] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.422914  8938 raft_consensus.cc:359] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.423681  8938 raft_consensus.cc:385] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.423995  8938 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.424660  8938 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.428929  8938 ts_tablet_manager.cc:1434] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:59:27.429682  8940 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.430090  8940 ts_tablet_manager.cc:1403] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.008s
I20260709 12:59:27.432644  8940 raft_consensus.cc:359] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.433220  8940 raft_consensus.cc:385] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.433481  8940 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.434047  8940 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.434870  8938 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.441972  8940 ts_tablet_manager.cc:1434] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:27.443665  8940 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.443712  8939 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.443754  9284 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.444327  8939 ts_tablet_manager.cc:1403] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 12:59:27.449759  8938 tablet_bootstrap.cc:654] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.449764  8940 tablet_bootstrap.cc:654] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.450580  8939 raft_consensus.cc:359] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.451287  8939 raft_consensus.cc:385] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.451576  8939 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.452320  8939 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.454331  8939 ts_tablet_manager.cc:1434] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:27.455212  8939 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.468916  8940 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.470247  8940 ts_tablet_manager.cc:1403] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.007s
I20260709 12:59:27.472409  9201 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.473223  8939 tablet_bootstrap.cc:654] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.472859  9201 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.473152  8940 raft_consensus.cc:359] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.473996  8940 raft_consensus.cc:385] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.474258  8940 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.474968  8940 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.475342  9201 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.476823  8940 ts_tablet_manager.cc:1434] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:27.477252  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.477844  8820 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:27.477806  8940 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.479507  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.480566  9190 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.481027  9190 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.481421  9190 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.482221  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.482842  8895 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:27.489497  8938 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.490075  8938 ts_tablet_manager.cc:1403] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.002s
I20260709 12:59:27.487541  9223 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.493353  9223 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.499066  8938 raft_consensus.cc:359] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.499960  8938 raft_consensus.cc:385] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.500203  8938 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.502501  9223 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.506546  8938 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.510213  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.510831  8744 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.511137  9190 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.513552  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.514464  8895 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.514819  8789 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.519163  8938 ts_tablet_manager.cc:1434] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.002s
I20260709 12:59:27.518581  9223 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.521692  9223 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.522146  9223 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.532095  9190 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.535169  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:27.535774  8820 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.536913  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.537535  8895 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.539039  9199 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.540228  8938 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.547869  8895 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:27.548195  8939 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.548667  8939 ts_tablet_manager.cc:1403] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.000s
I20260709 12:59:27.549611  8712 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.555753  8820 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:27.558961  8940 tablet_bootstrap.cc:654] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.551923  9201 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.559710  9201 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:27.569007  9223 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.569324  9201 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.578107  9197 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.588896  9223 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.589094  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.589751  8894 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.599658  9113 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.600245  8894 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.600145  9113 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.601269  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:27.602131  9113 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.603082  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.603667  8893 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:27.604766  8938 tablet_bootstrap.cc:654] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.604679  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.605167  9223 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.605405  9113 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.605798  9113 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.606127  9113 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.606866  9223 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:27.607573  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.607792  9223 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.611231  9197 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.610853  8939 raft_consensus.cc:359] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.616745  8939 raft_consensus.cc:385] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.617031  8939 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.618469  9113 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.624277  9113 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:27.609375  8715 leader_election.cc:343] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.625224  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.634809  8893 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.640945  8893 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:27.641902  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.644878  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:27.645442  9113 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.645983  9113 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:27.646459  8646 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.622175  8940 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.656085  8940 ts_tablet_manager.cc:1403] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.179s	user 0.010s	sys 0.004s
I20260709 12:59:27.646819  9113 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.647753  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:27.659487  8940 raft_consensus.cc:359] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.660256  8940 raft_consensus.cc:385] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.660544  8940 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
W20260709 12:59:27.655094  8715 leader_election.cc:343] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.617835  8939 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.666275  8940 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.668028  8939 ts_tablet_manager.cc:1434] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.119s	user 0.005s	sys 0.002s
I20260709 12:59:27.668509  8744 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.669378  8939 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.657907  8645 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.677347  8940 ts_tablet_manager.cc:1434] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:59:27.678048  8938 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.678463  8940 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.678548  8938 ts_tablet_manager.cc:1403] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.139s	user 0.011s	sys 0.002s
I20260709 12:59:27.679914  9108 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.681857  8744 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.693207  8938 raft_consensus.cc:359] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.694290  8938 raft_consensus.cc:385] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.694669  8938 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.695657  8938 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.700663  8938 ts_tablet_manager.cc:1434] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 12:59:27.702054  9223 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.712055  9223 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.715934  8820 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.721645  8939 tablet_bootstrap.cc:654] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.715971  8938 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.721913  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.724957  9223 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.738575  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.744045  9108 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.748255  8789 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:27.749702  9223 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.759079  8938 tablet_bootstrap.cc:654] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.769690  8940 tablet_bootstrap.cc:654] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.774533  9113 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.774771  8939 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.775220  8939 ts_tablet_manager.cc:1403] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.106s	user 0.006s	sys 0.007s
I20260709 12:59:27.775069  9113 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.777921  8939 raft_consensus.cc:359] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.778529  8939 raft_consensus.cc:385] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.778784  8939 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.767338  8646 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.779476  8939 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.783982  9113 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.784477  8939 ts_tablet_manager.cc:1434] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:59:27.785072  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.785513  8939 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
W20260709 12:59:27.786501  8712 leader_election.cc:343] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.789067  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.792516  9325 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.792959  9325 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.795830  9185 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.796312  9185 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.799825  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.801741  8894 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:27.802961  8712 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.803083  9325 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.803984  9113 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.804119  8939 tablet_bootstrap.cc:654] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:27.804417  8715 leader_election.cc:343] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.804860  8715 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:27.804399  9113 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.805707  9113 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.805734  9188 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.807050  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.807695  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:27.817727  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.818399  8895 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.822189  8787 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:27.822593  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.822999  9199 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.823798  8741 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.824767  8940 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.825193  8940 ts_tablet_manager.cc:1403] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.000s
I20260709 12:59:27.827734  8940 raft_consensus.cc:359] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.800467  9185 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.823945  9199 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.829191  8940 raft_consensus.cc:385] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.829157  9199 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.829537  8940 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.830336  8940 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.836903  8938 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:27.837390  8938 ts_tablet_manager.cc:1403] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.015s
I20260709 12:59:27.840255  8938 raft_consensus.cc:359] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.840451  9199 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.840917  8938 raft_consensus.cc:385] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.841189  8938 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.841729  9325 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.842067  8938 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.842162  9325 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.844419  9325 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.844877  8938 ts_tablet_manager.cc:1434] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:27.846300  8938 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:27.846158  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.846711  8894 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.848591  8787 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:27.849524  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:27.850091  8940 ts_tablet_manager.cc:1434] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:59:27.849138  9113 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.851267  8940 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:27.851323  9325 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.851735  9325 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.852056  9325 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.855193  8741 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:27.856899  9199 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.858445  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.859151  8894 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.903173  8939 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:27.903720  8939 ts_tablet_manager.cc:1403] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.011s
I20260709 12:59:27.906168  8939 raft_consensus.cc:359] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.906826  8939 raft_consensus.cc:385] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.907065  8939 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.912948  8894 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.913416  8939 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.915686  8787 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:27.916527  9199 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.917151  9199 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:27.917928  9199 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.917929  9113 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.923681  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:27.924188  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:27.924350  8741 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.924719  8822 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.930687  8939 ts_tablet_manager.cc:1434] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 12:59:27.931721  8939 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:27.933459  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:27.938659  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.939200  8894 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.941653  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:27.941776  8741 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.942345  9185 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.942926  9185 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:27.918077  9325 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.943708  9185 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.947767  9190 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.948199  9190 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:27.950083  9190 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.954953  8940 tablet_bootstrap.cc:654] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.960388  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:27.957916  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:27.961104  8741 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.964862  8938 tablet_bootstrap.cc:654] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.955799  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:27.966974  9325 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:27.971985  8939 tablet_bootstrap.cc:654] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:27.972390  8715 leader_election.cc:343] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.976999  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:27.977768  8895 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:27.984252  8712 leader_election.cc:343] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.985996  8940 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:27.986466  8940 ts_tablet_manager.cc:1403] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.136s	user 0.011s	sys 0.000s
I20260709 12:59:27.986370  8712 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:27.987457  9201 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.988762  8894 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:27.998662  8741 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:27.989177  8940 raft_consensus.cc:359] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.004802  8940 raft_consensus.cc:385] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.005074  8940 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.005348  8644 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.006992  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.009565  9325 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.010254  9325 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:28.013110  8940 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.013263  9325 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.008492  9223 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.014843  8940 ts_tablet_manager.cc:1434] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:59:28.026615  8939 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:28.027096  8939 ts_tablet_manager.cc:1403] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.000s
I20260709 12:59:28.029112  8940 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:28.020223  8895 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:28.113966  8646 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.122805  8939 raft_consensus.cc:359] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.123714  8939 raft_consensus.cc:385] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.124058  8939 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.126355  8939 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.118582  9275 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.135771  9190 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:28.153787  8695 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:28.167073  8938 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.167615  8938 ts_tablet_manager.cc:1403] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.322s	user 0.018s	sys 0.010s
I20260709 12:59:28.141983  9190 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.138518  9275 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.178802  9275 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.193023  8939 ts_tablet_manager.cc:1434] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.166s	user 0.008s	sys 0.006s
I20260709 12:59:28.194201  8939 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Bootstrap starting.
I20260709 12:59:28.195380  8938 raft_consensus.cc:359] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.186633  9349 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.199290  8938 raft_consensus.cc:385] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.203222  8938 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.205031  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.214617  8938 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.198644  9350 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 12:59:28.232093  8940 tablet_bootstrap.cc:654] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.205864  9349 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.250378  9108 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.251045  9108 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.254663  9108 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:28.257076  8938 ts_tablet_manager.cc:1434] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.089s	user 0.009s	sys 0.001s
I20260709 12:59:28.261619  8821 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.273481  9349 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.256304  9348 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:59:28.264523  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:28.282342  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:28.282713  8939 tablet_bootstrap.cc:654] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.282718  8938 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:28.289198  8892 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 1: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 12:59:28.315155  8938 tablet_bootstrap.cc:654] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.318063  8940 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:28.318732  8940 ts_tablet_manager.cc:1403] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.294s	user 0.017s	sys 0.019s
I20260709 12:59:28.319258  9190 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.327064  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.327800  8822 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.347373  8939 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: No bootstrap required, opened a new log
I20260709 12:59:28.349313  8939 ts_tablet_manager.cc:1403] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Time spent bootstrapping tablet: real 0.155s	user 0.021s	sys 0.004s
I20260709 12:59:28.352365  8938 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.352946  8938 ts_tablet_manager.cc:1403] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.009s
I20260709 12:59:28.349989  8940 raft_consensus.cc:359] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.353463  8940 raft_consensus.cc:385] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.353802  8940 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.354523  8940 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.356715  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:28.357947  8938 raft_consensus.cc:359] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.359022  8938 raft_consensus.cc:385] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.359275  8938 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.359906  8938 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.360674  8894 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:28.360952  8940 ts_tablet_manager.cc:1434] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.002s
I20260709 12:59:28.362051  8940 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:28.366379  8938 ts_tablet_manager.cc:1434] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:59:28.367275  8938 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:28.370005  8940 tablet_bootstrap.cc:654] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.373808  8938 tablet_bootstrap.cc:654] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.374782  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:28.375682  9344 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.376019  9344 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.376397  9344 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.382663  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.383467  8821 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 0.
I20260709 12:59:28.395394  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.396088  8741 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.397781  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.384805  8938 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.399427  8938 ts_tablet_manager.cc:1403] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.000s
W20260709 12:59:28.386339  8712 leader_election.cc:343] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.385179  8939 raft_consensus.cc:359] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.399914  8939 raft_consensus.cc:385] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.400206  8939 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.400789  8939 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.402207  8939 ts_tablet_manager.cc:1434] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.000s
I20260709 12:59:28.404793  9351 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.408082  9345 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.408666  9345 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.409160  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.410908  9275 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.411322  9275 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.411346  9345 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.411712  9275 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.413311  8938 raft_consensus.cc:359] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.413957  8938 raft_consensus.cc:385] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.425674  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:28.433902  9344 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.436962  9355 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.437490  9355 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.414263  8938 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.439461  9355 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.445262  8940 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:28.449928  8940 ts_tablet_manager.cc:1403] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.088s	user 0.040s	sys 0.032s
W20260709 12:59:28.445566  8715 leader_election.cc:343] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.424860  9275 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.452732  9275 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.454583  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 12:59:28.455400  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57478.2us
I20260709 12:59:28.456121  9355 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.460554  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:28.448477  8712 leader_election.cc:343] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.471024  9113 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:28.471058  8712 leader_election.cc:343] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.479534  9355 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.484702  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:28.485486  8821 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.489395  8646 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.492810  8821 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:28.494892  9358 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.497977  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:28.494978  8940 raft_consensus.cc:359] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.498769  8940 raft_consensus.cc:385] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.499061  8940 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.499429  9313 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.499939  9313 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:28.499682  8940 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.500741  9313 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.501926  8940 ts_tablet_manager.cc:1434] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 12:59:28.502910  8940 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:28.505149  8938 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.507424  8938 ts_tablet_manager.cc:1434] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.000s
I20260709 12:59:28.508607  8940 tablet_bootstrap.cc:654] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.496930  9344 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:28.522324  8789 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:28.523387  9196 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.528904  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.532666  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.533324  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:28.534003  8898 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.538061  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:28.538703  8819 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.539625  8940 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:28.540098  8940 ts_tablet_manager.cc:1403] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.008s
I20260709 12:59:28.542616  8940 raft_consensus.cc:359] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.544193  8940 raft_consensus.cc:385] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.544457  8940 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.508616  8938 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:28.545087  8940 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.551947  8940 ts_tablet_manager.cc:1434] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:59:28.552861  8940 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:28.558755  8940 tablet_bootstrap.cc:654] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.567176  8819 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:28.568666  8898 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 12:59:28.569603  8645 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:28.573877  8938 tablet_bootstrap.cc:654] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.575929  9355 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.584934  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:28.603391  8742 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.606312  8940 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:28.606761  8940 ts_tablet_manager.cc:1403] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.054s	user 0.025s	sys 0.012s
I20260709 12:59:28.611384  8742 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:28.612167  8940 raft_consensus.cc:359] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.612874  8940 raft_consensus.cc:385] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.613152  8940 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.614920  9196 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.615424  9196 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.617309  8940 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.620407  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.624362  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 12:59:28.625651  8787 leader_election.cc:343] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.626497  8938 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.626966  8938 ts_tablet_manager.cc:1403] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.012s
I20260709 12:59:28.628692  9355 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.629359  9355 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:28.629475  8938 raft_consensus.cc:359] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.630143  9355 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.631103  8940 ts_tablet_manager.cc:1434] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:59:28.630151  8938 raft_consensus.cc:385] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.633006  8938 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.632097  8940 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Bootstrap starting.
I20260709 12:59:28.637029  8938 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.641276  9196 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.642082  8938 ts_tablet_manager.cc:1434] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:59:28.643146  8938 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
I20260709 12:59:28.646422  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.647058  8740 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:28.649077  8938 tablet_bootstrap.cc:654] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.653941  8940 tablet_bootstrap.cc:654] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.669262  8940 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: No bootstrap required, opened a new log
I20260709 12:59:28.670207  8940 ts_tablet_manager.cc:1403] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
I20260709 12:59:28.672992  8940 raft_consensus.cc:359] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.673781  8940 raft_consensus.cc:385] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.674062  8940 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.674831  8940 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.682746  8938 tablet_bootstrap.cc:492] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.683274  8938 ts_tablet_manager.cc:1403] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 12:59:28.684221  9196 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.688124  8940 ts_tablet_manager.cc:1434] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:59:28.688684  8938 raft_consensus.cc:359] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.689445  8938 raft_consensus.cc:385] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.689719  8938 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.690374  8938 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.691813  9275 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.692378  8938 ts_tablet_manager.cc:1434] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
W20260709 12:59:28.695837  8715 leader_election.cc:343] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.695554  8645 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.696244  8715 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:28.696839  9275 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.698477  8938 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Bootstrap starting.
W20260709 12:59:28.702581  8715 leader_election.cc:343] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.702996  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:28.703795  9275 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.704154  8938 tablet_bootstrap.cc:654] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.706200  9353 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.714967  8938 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: No bootstrap required, opened a new log
I20260709 12:59:28.715468  8938 ts_tablet_manager.cc:1403] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.008s
I20260709 12:59:28.718163  8938 raft_consensus.cc:359] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.718813  8938 raft_consensus.cc:385] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.719079  8938 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.719624  8938 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.721783  8938 ts_tablet_manager.cc:1434] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:28.725947  9196 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.728138  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:28.729167  9196 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.729576  9196 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.729892  9196 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.744050  9344 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.744549  9344 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.746536  9344 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:28.739413  9196 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.748019  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.748615  8819 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.750648  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:28.751642  9313 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.752525  9313 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.752934  9313 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.757082  9353 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.757615  9353 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.760512  9353 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.761181  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:28.761813  8747 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.761227  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 12:59:28.764611  8895 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:28.765908  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:28.767733  9325 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.768095  9325 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.768419  9325 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.769459  8747 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:28.770618  9196 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.774370  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.775842  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.775789  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.776503  8747 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.776592  8742 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 0.
I20260709 12:59:28.776664  9284 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.777246  9284 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:28.778596  9284 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.780553  9325 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.783933  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:28.784729  8896 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.800757  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:28.801497  8742 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.810830  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:28.811439  8894 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.827085  8646 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.808138  9313 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.814102  9325 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:28.833920  9313 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:28.842290  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:28.846262  8749 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.863494  8742 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:28.864427  8894 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:28.865564  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:28.869179  9353 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.869647  9353 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:28.870401  9353 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.855326  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:28.871480  8749 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:28.871668  8821 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.874361  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.875496  9344 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.876523  8896 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:28.875962  9344 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:28.878279  9344 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.904904  8821 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:28.909734  8646 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.920003  8645 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:28.985888  9344 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.986318  9344 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:28.989346  9344 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:28.989383  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 12:59:28.990159  8744 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.992955  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:28.993845  9344 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.990110  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:28.994256  9344 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.995079  8821 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 0.
I20260709 12:59:28.995889  9344 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.003139  9344 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:29.009191  9344 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:29.009883  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:29.010138  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:29.010542  8744 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.010710  8821 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.017256  8821 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:29.018078  8744 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 12:59:29.018307  8864 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:29.019507  9344 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.019989  9344 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:29.020819  9344 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:29.032639  9344 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.033097  8646 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:29.048765  9355 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.053035  9272 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.080998  9275 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.235556  9284 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.247959  9353 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.326318  9344 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.346916  9272 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.381114  9284 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.394341  9325 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.547793  9272 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.568207  9108 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:30.133154  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:30.164945  9403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:30.182061  9404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:30.186543  7638 server_base.cc:1061] running on GCE node
W20260709 12:59:30.189781  9406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:30.191128  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:30.191443  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:30.191679  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783601970191660 us; error 0 us; skew 500 ppm
I20260709 12:59:30.192445  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:30.195914  7638 webserver.cc:533] Webserver started at http://127.7.117.132:34493/ using document root <none> and password file <none>
I20260709 12:59:30.196619  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:30.196868  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:30.197258  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:30.198967  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-3-root/instance:
uuid: "112105225cc140459688be5d31dcb071"
format_stamp: "Formatted at 2026-07-09 12:59:30 on dist-test-slave-ndzg"
I20260709 12:59:30.207322  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:59:30.212455  9411 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:30.213327  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:59:30.213707  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
uuid: "112105225cc140459688be5d31dcb071"
format_stamp: "Formatted at 2026-07-09 12:59:30 on dist-test-slave-ndzg"
I20260709 12:59:30.214080  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:30.228889  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:30.230728  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:30.232846  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:30.242918  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:30.243185  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:30.243492  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:30.243718  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:30.418507  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.132:38693
I20260709 12:59:30.418746  9473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.132:38693 every 8 connection(s)
I20260709 12:59:30.436116  7638 auto_leader_rebalancer-test.cc:384] add a tserver: 112105225cc140459688be5d31dcb071
I20260709 12:59:30.459102  9474 heartbeater.cc:344] Connected to a master server at 127.7.117.190:45111
I20260709 12:59:30.459664  9474 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:30.460500  9474 heartbeater.cc:507] Master 127.7.117.190:45111 requested a full tablet report, sending...
I20260709 12:59:30.462339  8645 ts_manager.cc:194] Registered new tserver with Master: 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:30.464001  8645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52550
I20260709 12:59:31.437989  8744 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:31.460173  8894 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.462998  9478 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:31.466001  8819 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.467674  9474 heartbeater.cc:499] Master 127.7.117.190:45111 was elected leader, sending a full tablet report...
I20260709 12:59:31.469615  9479 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:31.483141  8894 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.480688  9478 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 12:59:31.500774  8713 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ac343934349411088cbf7a05932c2a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.501711  8820 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.636790  8645 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:31.726204  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 73116.4us
I20260709 12:59:31.791594  8896 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:31.842691  8821 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.844894  9313 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.851400  8742 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.852900  9313 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.861227  9500 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.864039  8821 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.880142  8898 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:31.893465  8645 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.891616  8742 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.903491  8821 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.905006  9313 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:31.911350  8862 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc45d3a2731b49daacb16548dc742815. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.921878  8742 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:31.922480  8862 consensus_peers.cc:597] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dada29d2380a453682f5dfef36dcf407. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.923213  9500 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.932694  9480 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.948019  8896 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:31.950033  8744 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.953423  8821 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:31.955437  8645 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.969750  8821 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:31.970118  8862 consensus_peers.cc:597] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db96d769069547e38edb50dbebdbe00a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:31.971227  9502 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.976473  8744 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:31.978729  9502 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:31.999578  9480 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:32.001138  8744 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:32.021148  8644 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.022900  8821 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:32.078514  9520 ts_tablet_manager.cc:933] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:32.081890  9520 tablet_copy_client.cc:323] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:32.121793  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 4ac343934349411088cbf7a05932c2a5 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:32.122471  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 4ac343934349411088cbf7a05932c2a5 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-4ac343934349411088cbf7a05932c2a5
I20260709 12:59:32.130880  8759 tablet_copy_source_session.cc:215] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.135510  9520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac343934349411088cbf7a05932c2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.159127  9520 tablet_copy_client.cc:806] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.159754  9520 tablet_copy_client.cc:670] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.164116  9520 tablet_copy_client.cc:538] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.173158  9520 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:32.206228  9520 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.207157  9520 tablet_bootstrap.cc:492] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:32.207798  9520 ts_tablet_manager.cc:1403] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.035s	user 0.027s	sys 0.004s
I20260709 12:59:32.210592  9520 raft_consensus.cc:359] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.211460  9520 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:32.212086  9520 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.221231  9520 ts_tablet_manager.cc:1434] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 12:59:32.225956  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-4ac343934349411088cbf7a05932c2a5 received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:32.226444  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-4ac343934349411088cbf7a05932c2a5 on tablet 4ac343934349411088cbf7a05932c2a5 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:32.253912  9520 ts_tablet_manager.cc:933] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:32.257632  9520 tablet_copy_client.cc:323] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:32.266534  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet fc45d3a2731b49daacb16548dc742815 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:32.267032  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet fc45d3a2731b49daacb16548dc742815 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-fc45d3a2731b49daacb16548dc742815
I20260709 12:59:32.277348  8908 tablet_copy_source_session.cc:215] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.280174  9520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc45d3a2731b49daacb16548dc742815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.292874  9520 tablet_copy_client.cc:806] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.293444  9520 tablet_copy_client.cc:670] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.303871  9520 tablet_copy_client.cc:538] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.316334  9520 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:32.494242  9528 ts_tablet_manager.cc:933] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:32.524762  9530 ts_tablet_manager.cc:933] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:32.527168  9530 tablet_copy_client.cc:323] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:32.528843  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet db96d769069547e38edb50dbebdbe00a from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:32.530058  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet db96d769069547e38edb50dbebdbe00a from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-db96d769069547e38edb50dbebdbe00a
I20260709 12:59:32.543919  8908 tablet_copy_source_session.cc:215] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:59:32.544844  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44907.1us
I20260709 12:59:32.590008  9520 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.590934  9520 tablet_bootstrap.cc:492] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:32.591575  9520 ts_tablet_manager.cc:1403] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.275s	user 0.085s	sys 0.147s
I20260709 12:59:32.594532  9520 raft_consensus.cc:359] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.595553  9520 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:32.596230  9520 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.606207  9530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db96d769069547e38edb50dbebdbe00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.617555  9520 ts_tablet_manager.cc:1434] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.001s
I20260709 12:59:32.618726  9528 tablet_copy_client.cc:323] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:32.620375  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet dada29d2380a453682f5dfef36dcf407 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:32.620903  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet dada29d2380a453682f5dfef36dcf407 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-dada29d2380a453682f5dfef36dcf407
I20260709 12:59:32.639773  9530 tablet_copy_client.cc:806] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.640384  9530 tablet_copy_client.cc:670] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.641230  8907 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-fc45d3a2731b49daacb16548dc742815 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:32.641744  8907 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-fc45d3a2731b49daacb16548dc742815 on tablet fc45d3a2731b49daacb16548dc742815 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:32.643748  9530 tablet_copy_client.cc:538] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.651120  8908 tablet_copy_source_session.cc:215] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.655975  9528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dada29d2380a453682f5dfef36dcf407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.651858  9530 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:32.676900  9449 raft_consensus.cc:1217] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.708794  9528 tablet_copy_client.cc:806] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.709414  9528 tablet_copy_client.cc:670] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.715426  9528 tablet_copy_client.cc:538] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.719135  9530 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.720067  9530 tablet_bootstrap.cc:492] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:32.720721  9530 ts_tablet_manager.cc:1403] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.069s	user 0.021s	sys 0.000s
I20260709 12:59:32.724220  9528 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:32.724910  9530 raft_consensus.cc:359] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.725755  9530 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:32.726379  9530 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.730603  9530 ts_tablet_manager.cc:1434] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:32.735037  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-db96d769069547e38edb50dbebdbe00a received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:32.735641  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-db96d769069547e38edb50dbebdbe00a on tablet db96d769069547e38edb50dbebdbe00a with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:32.787571  9528 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.788488  9528 tablet_bootstrap.cc:492] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:32.789186  9528 ts_tablet_manager.cc:1403] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.006s
I20260709 12:59:32.791982  9528 raft_consensus.cc:359] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.792826  9528 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:32.793485  9528 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:32.796197  9528 ts_tablet_manager.cc:1434] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:32.803340  9449 raft_consensus.cc:1217] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.805862  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-dada29d2380a453682f5dfef36dcf407 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:32.806313  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-dada29d2380a453682f5dfef36dcf407 on tablet dada29d2380a453682f5dfef36dcf407 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:33.033437  9449 raft_consensus.cc:1217] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:33.105276  9449 raft_consensus.cc:1217] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:33.114610  9533 raft_consensus.cc:1064] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:33.117833  9533 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.144800  9449 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.146895  9505 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.163185  8896 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.164638  9505 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.178778  9447 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.187517  9532 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.199648  8896 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.209405  8645 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.226214  8820 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.275761  9533 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.315569  8819 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.360772  9480 raft_consensus.cc:1064] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:33.363189  9480 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.369714  8819 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.372265  9447 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.372408  9536 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.373746  9480 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.375710  8744 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.377619  9536 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.415478  9480 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.417488  8822 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.424396  8744 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.442747  8645 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.448725  9449 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.491842  8896 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.512871  9538 raft_consensus.cc:1064] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:33.515100  9449 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.519922  9538 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.523420  8744 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.525170  9536 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.532100  9536 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.550333  9536 raft_consensus.cc:1064] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:33.552765  9536 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.553995  9313 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.558598  8744 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.561599  8822 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.563843  9449 raft_consensus.cc:2955] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.567203  9480 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.618973  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fc45d3a2731b49daacb16548dc742815 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:33.630225  8645 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.646871  8822 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.651615  9313 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.660121  9447 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.661545  9313 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.668298  9447 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.694505  8747 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.702098  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet fc45d3a2731b49daacb16548dc742815 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:33.670099  8741 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.706871  9553 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:59:33.719478  9562 tablet_replica.cc:333] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:33.720532  9562 raft_consensus.cc:2243] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:33.685501  9536 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:59:33.743567  9562 raft_consensus.cc:2272] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:33.751521  9313 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:59:33.757231  9556 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:59:33.788378  9562 ts_tablet_manager.cc:1916] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:33.801618  9554 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.825268  8822 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.838675  8646 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.859812  8744 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.841209  9448 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.881196  8892 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.897214  9313 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.904719  9447 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.908588  8818 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.908763  9554 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.909396  9447 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.915594  9554 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.916723  8741 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.924751  9480 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.933038  8819 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.934806  9448 raft_consensus.cc:2955] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.938364  9562 ts_tablet_manager.cc:1929] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:33.938853  9562 log.cc:1199] T fc45d3a2731b49daacb16548dc742815 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/fc45d3a2731b49daacb16548dc742815
I20260709 12:59:33.940558  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet fc45d3a2731b49daacb16548dc742815 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:33.940608  8744 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.942022  8646 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.954758  8818 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.958871  8897 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:33.960283  9447 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.964172  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dada29d2380a453682f5dfef36dcf407 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:33.970726  9449 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.972229  8819 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.973271  9558 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.975396  9505 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.977627  9554 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.967214  8646 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.984629  9533 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.990650  8819 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.991986  9449 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:33.992662  9536 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:34.002970  9480 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:34.004222  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet dada29d2380a453682f5dfef36dcf407 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:34.015638  9447 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:34.019388  8820 raft_consensus.cc:2955] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:34.029320  8819 raft_consensus.cc:2955] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:34.035138  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ac343934349411088cbf7a05932c2a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:34.044973  8646 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.052707  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet db96d769069547e38edb50dbebdbe00a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:34.066496  9567 tablet_replica.cc:333] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:34.067510  9567 raft_consensus.cc:2243] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.069576  9567 raft_consensus.cc:2272] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.071870  8647 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.074990  9567 ts_tablet_manager.cc:1916] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.084964  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ac343934349411088cbf7a05932c2a5 with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:34.094863  9567 ts_tablet_manager.cc:1929] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.095295  9567 log.cc:1199] T dada29d2380a453682f5dfef36dcf407 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/dada29d2380a453682f5dfef36dcf407
I20260709 12:59:34.096956  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet dada29d2380a453682f5dfef36dcf407 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:34.103513  9568 tablet_replica.cc:333] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:34.107690  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet db96d769069547e38edb50dbebdbe00a with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:34.108976  9567 tablet_replica.cc:333] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:34.119510  9567 raft_consensus.cc:2243] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.120136  9567 raft_consensus.cc:2272] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.124405  9567 ts_tablet_manager.cc:1916] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.126443  9568 raft_consensus.cc:2243] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.127179  9568 raft_consensus.cc:2272] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.130470  9568 ts_tablet_manager.cc:1916] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.138250  9567 ts_tablet_manager.cc:1929] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.138618  9567 log.cc:1199] T db96d769069547e38edb50dbebdbe00a P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/db96d769069547e38edb50dbebdbe00a
I20260709 12:59:34.139886  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet db96d769069547e38edb50dbebdbe00a (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:34.147589  9568 ts_tablet_manager.cc:1929] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.148030  9568 log.cc:1199] T 4ac343934349411088cbf7a05932c2a5 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/4ac343934349411088cbf7a05932c2a5
I20260709 12:59:34.149600  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 4ac343934349411088cbf7a05932c2a5 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:35.225951  8747 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:35.233633  8713 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5bf6ec0e6a745d0967928a3361eb8d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.234407  8896 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.236830  8819 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.236608  9533 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.243285  9533 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.250418  9506 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.258123  8896 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.264120  8820 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.279560  8643 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:35.278486  8747 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:35.301308  8713 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a0644aef66444b58e8633da6993535c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.303730  8896 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.304387  8820 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.305483  9533 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.306195  9506 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.319686  9533 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.322935  8896 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.332705  8897 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:35.350571  8862 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 011b370821004185a003be210a525c01. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.350363  8647 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.354198  8819 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.355650  8747 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.356137  9536 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.365912  9536 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.381803  9570 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.391083  8819 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.403784  8647 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:35.425273  8747 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.411990  8820 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.526678  8819 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.533582  8897 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.534898  9325 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.541998  8747 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.543527  9564 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.549813  9564 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.554420  8897 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 12:59:35.569252  8788 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53864dd6ef544420915d557a63d5cd91. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.577272  8647 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.594359  8747 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:35.673609  9594 ts_tablet_manager.cc:933] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:35.680404  9594 tablet_copy_client.cc:323] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:35.682089  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet c5bf6ec0e6a745d0967928a3361eb8d8 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:35.682709  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet c5bf6ec0e6a745d0967928a3361eb8d8 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-c5bf6ec0e6a745d0967928a3361eb8d8
I20260709 12:59:35.690770  8759 tablet_copy_source_session.cc:215] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.694195  9594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bf6ec0e6a745d0967928a3361eb8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.711012  9594 tablet_copy_client.cc:806] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.712725  9594 tablet_copy_client.cc:670] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.716684  9594 tablet_copy_client.cc:538] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.726437  9594 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:35.756217  9594 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.757125  9594 tablet_bootstrap.cc:492] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:35.757805  9594 ts_tablet_manager.cc:1403] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.003s
I20260709 12:59:35.760633  9594 raft_consensus.cc:359] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.761476  9594 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:35.771166  9594 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.802255  9597 ts_tablet_manager.cc:933] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:35.804935  9597 tablet_copy_client.cc:323] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:35.808257  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 1a0644aef66444b58e8633da6993535c from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:35.808816  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 1a0644aef66444b58e8633da6993535c from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-1a0644aef66444b58e8633da6993535c
I20260709 12:59:35.816118  8759 tablet_copy_source_session.cc:215] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.819815  9597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a0644aef66444b58e8633da6993535c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.822978  9594 ts_tablet_manager.cc:1434] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.010s
I20260709 12:59:35.824728  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-c5bf6ec0e6a745d0967928a3361eb8d8 received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:35.825166  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-c5bf6ec0e6a745d0967928a3361eb8d8 on tablet c5bf6ec0e6a745d0967928a3361eb8d8 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:35.837280  9594 ts_tablet_manager.cc:933] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:35.841784  9597 tablet_copy_client.cc:806] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.842445  9597 tablet_copy_client.cc:670] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.849095  9597 tablet_copy_client.cc:538] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.849711  9594 tablet_copy_client.cc:323] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:35.852223  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 011b370821004185a003be210a525c01 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:35.852792  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 011b370821004185a003be210a525c01 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-011b370821004185a003be210a525c01
I20260709 12:59:35.858673  9597 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:35.862241  8908 tablet_copy_source_session.cc:215] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:35.865672  9594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011b370821004185a003be210a525c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:35.882125  9594 tablet_copy_client.cc:806] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:35.882702  9594 tablet_copy_client.cc:670] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:35.886643  9594 tablet_copy_client.cc:538] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:35.896076  9594 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:35.912526  9597 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.913504  9597 tablet_bootstrap.cc:492] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:35.914175  9597 ts_tablet_manager.cc:1403] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.014s
I20260709 12:59:35.918807  9597 raft_consensus.cc:359] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.919898  9597 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:35.920733  9597 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.931322  9597 ts_tablet_manager.cc:1434] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:59:35.937207  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-1a0644aef66444b58e8633da6993535c received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:35.937861  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-1a0644aef66444b58e8633da6993535c on tablet 1a0644aef66444b58e8633da6993535c with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:35.951933  9594 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:35.952898  9594 tablet_bootstrap.cc:492] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:35.953616  9594 ts_tablet_manager.cc:1403] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.019s
I20260709 12:59:35.956776  9594 raft_consensus.cc:359] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.958214  9594 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:35.958869  9594 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:35.961175  9594 ts_tablet_manager.cc:1434] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:59:35.964236  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-011b370821004185a003be210a525c01 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:35.964651  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-011b370821004185a003be210a525c01 on tablet 011b370821004185a003be210a525c01 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:36.089586  9594 ts_tablet_manager.cc:933] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:36.104118  9594 tablet_copy_client.cc:323] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:36.146512  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 53864dd6ef544420915d557a63d5cd91 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:36.147105  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 53864dd6ef544420915d557a63d5cd91 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-53864dd6ef544420915d557a63d5cd91
I20260709 12:59:36.156605  8834 tablet_copy_source_session.cc:215] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:36.159781  9594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53864dd6ef544420915d557a63d5cd91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:36.174541  9594 tablet_copy_client.cc:806] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:36.175060  9594 tablet_copy_client.cc:670] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:36.178684  9594 tablet_copy_client.cc:538] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:36.179095  9447 raft_consensus.cc:1217] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.188341  9594 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:36.216144  9594 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:36.217131  9594 tablet_bootstrap.cc:492] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:36.217754  9594 ts_tablet_manager.cc:1403] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.009s
I20260709 12:59:36.222036  9594 raft_consensus.cc:359] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:36.222878  9594 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:36.223433  9594 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:36.227432  9594 ts_tablet_manager.cc:1434] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 12:59:36.228981  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-53864dd6ef544420915d557a63d5cd91 received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:36.229430  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-53864dd6ef544420915d557a63d5cd91 on tablet 53864dd6ef544420915d557a63d5cd91 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:36.263938  9447 raft_consensus.cc:1217] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.418579  9447 raft_consensus.cc:1217] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.524989  9447 raft_consensus.cc:1217] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.632762  9506 raft_consensus.cc:1064] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:36.635128  9506 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:36.657105  9447 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.658030  8896 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.658993  9533 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.661229  9584 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.680349  8819 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.688184  9584 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.711580  9611 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.719002  8898 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.729776  9447 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.741259  8647 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.741859  8819 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.798055  8749 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:36.809278  9447 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:36.809834  8822 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:36.811703  9611 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:36.814510  9611 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:36.823273  8822 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.826548  9612 raft_consensus.cc:1064] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:36.826893  9449 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.829388  9612 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:36.820856  9611 raft_consensus.cc:2955] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.845535  8643 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.849866  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c5bf6ec0e6a745d0967928a3361eb8d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:36.858081  8896 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.858944  8819 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.859416  9449 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:36.860117  9612 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.861300  9533 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:36.862674  9582 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:36.888808  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet c5bf6ec0e6a745d0967928a3361eb8d8 with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:36.899362  9625 tablet_replica.cc:333] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:36.909400  9625 raft_consensus.cc:2243] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:36.910171  9625 raft_consensus.cc:2272] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:36.918072  9625 ts_tablet_manager.cc:1916] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:36.925768  9582 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.929108  9607 raft_consensus.cc:1064] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:36.929630  8896 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.932282  9607 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:36.942705  9449 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.946930  9625 ts_tablet_manager.cc:1929] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:36.947443  9625 log.cc:1199] T c5bf6ec0e6a745d0967928a3361eb8d8 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/c5bf6ec0e6a745d0967928a3361eb8d8
I20260709 12:59:36.943300  8819 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:36.950780  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet c5bf6ec0e6a745d0967928a3361eb8d8 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:36.957428  8643 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.019757  8747 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:37.029276  8892 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.032984  9533 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.044827  9533 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.060385  8892 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.084162  9447 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.088075  8747 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.088454  8646 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.089540  9570 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.090071  9447 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.091483  9612 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.095731  9536 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.099777  8819 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.101154  9536 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.102169  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a0644aef66444b58e8633da6993535c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.103868  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a0644aef66444b58e8633da6993535c with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:37.107153  9628 tablet_replica.cc:333] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:37.107240  9564 raft_consensus.cc:1064] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:37.107306  9447 raft_consensus.cc:2955] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.108547  9628 raft_consensus.cc:2243] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.109393  9628 raft_consensus.cc:2272] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.112458  9564 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:37.130434  9628 ts_tablet_manager.cc:1916] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.154313  9570 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.158993  8819 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.170118  8747 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.175316  9628 ts_tablet_manager.cc:1929] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.175843  9628 log.cc:1199] T 1a0644aef66444b58e8633da6993535c P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/1a0644aef66444b58e8633da6993535c
I20260709 12:59:37.180737  9448 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.189697  8646 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.193897  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 1a0644aef66444b58e8633da6993535c (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:37.199339  9448 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.202013  8896 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.203569  9604 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.207379  8747 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.209939  9564 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.212006  9564 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:59:37.219640  8896 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:37.230522  9448 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.233248  9607 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.248463  9449 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.246232  9618 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.253986  8749 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.256788  9607 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.262403  8741 raft_consensus.cc:2955] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.267867  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 011b370821004185a003be210a525c01 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.272531  9325 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.274852  8749 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.289671  8643 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:37.301162  9447 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.304913  8898 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.314584  8647 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:37.322544  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 011b370821004185a003be210a525c01 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:37.323462  9628 tablet_replica.cc:333] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:37.324667  9628 raft_consensus.cc:2243] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.325390  9628 raft_consensus.cc:2272] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.330885  9628 ts_tablet_manager.cc:1916] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.333853  8819 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:37.350915  9449 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.352538  9564 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.356981  8741 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.358296  9325 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.360535  9604 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.361672  9447 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.376788  8741 raft_consensus.cc:2955] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.382078  9628 ts_tablet_manager.cc:1929] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.382630  9628 log.cc:1199] T 011b370821004185a003be210a525c01 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/011b370821004185a003be210a525c01
I20260709 12:59:37.384608  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 011b370821004185a003be210a525c01 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:37.390190  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53864dd6ef544420915d557a63d5cd91 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.400323  8647 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:37.414009  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 53864dd6ef544420915d557a63d5cd91 with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:37.415061  9625 tablet_replica.cc:333] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:37.416414  9625 raft_consensus.cc:2243] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.417038  9625 raft_consensus.cc:2272] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.421408  9625 ts_tablet_manager.cc:1916] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.436530  9625 ts_tablet_manager.cc:1929] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.436869  9625 log.cc:1199] T 53864dd6ef544420915d557a63d5cd91 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/53864dd6ef544420915d557a63d5cd91
I20260709 12:59:37.438310  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 53864dd6ef544420915d557a63d5cd91 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:38.487687  8898 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:38.493924  8819 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.494732  8749 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.495225  9639 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:38.496156  9618 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:38.498936  8862 consensus_peers.cc:597] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33a0adf8736244d3b208cbc85700f0f9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.507884  8819 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.507802  9536 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.512008  8749 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.527712  8643 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.552778  8749 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:38.561977  8713 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 029b57162d72422bba468c7d86b67fca. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.566186  8896 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.567880  9582 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.568945  8819 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.570643  9653 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:38.586833  9584 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.596232  8822 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.598531  8896 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.600554  8897 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:38.605887  8647 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:38.643014  8823 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:38.643530  8862 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f192c6b771043a893d6fd420c9b9e7d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.644284  9645 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.645081  8749 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.646270  9645 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.652213  9639 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.653872  8823 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.655906  8749 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.670804  8644 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.675812  8823 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:38.682837  8749 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:38.683570  8788 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 234e3fa2a66f42f3a5a755f5f4adc858. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.684175  9660 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.687110  8897 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.688357  9660 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.714601  9564 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.721594  8741 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.739535  8647 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.763123  8897 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:38.967525  9663 ts_tablet_manager.cc:933] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:38.972441  9663 tablet_copy_client.cc:323] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:38.974224  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 029b57162d72422bba468c7d86b67fca from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:38.974697  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 029b57162d72422bba468c7d86b67fca from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-029b57162d72422bba468c7d86b67fca
I20260709 12:59:38.980361  8759 tablet_copy_source_session.cc:215] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:38.983172  9663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 029b57162d72422bba468c7d86b67fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:38.995034  9663 tablet_copy_client.cc:806] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:38.995589  9663 tablet_copy_client.cc:670] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:38.999372  9663 tablet_copy_client.cc:538] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.006798  9663 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:39.029495  9663 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.030426  9663 tablet_bootstrap.cc:492] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:39.031162  9663 ts_tablet_manager.cc:1403] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.006s
I20260709 12:59:39.032339  9666 ts_tablet_manager.cc:933] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:39.034502  9663 raft_consensus.cc:359] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.035415  9663 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:39.036118  9663 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.043848  9663 ts_tablet_manager.cc:1434] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.003s
I20260709 12:59:39.045681  9666 tablet_copy_client.cc:323] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:39.046145  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-029b57162d72422bba468c7d86b67fca received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:39.046630  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-029b57162d72422bba468c7d86b67fca on tablet 029b57162d72422bba468c7d86b67fca with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:39.048367  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 33a0adf8736244d3b208cbc85700f0f9 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:39.049021  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 33a0adf8736244d3b208cbc85700f0f9 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-33a0adf8736244d3b208cbc85700f0f9
I20260709 12:59:39.049790  9663 ts_tablet_manager.cc:933] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:39.074723  8908 tablet_copy_source_session.cc:215] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.079380  9666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0adf8736244d3b208cbc85700f0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.085824  9663 tablet_copy_client.cc:323] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:39.090217  9668 ts_tablet_manager.cc:933] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:39.093266  9668 tablet_copy_client.cc:323] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:39.093592  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 3f192c6b771043a893d6fd420c9b9e7d from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:39.094129  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 3f192c6b771043a893d6fd420c9b9e7d from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-3f192c6b771043a893d6fd420c9b9e7d
I20260709 12:59:39.095112  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 234e3fa2a66f42f3a5a755f5f4adc858 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:39.095646  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 234e3fa2a66f42f3a5a755f5f4adc858 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-234e3fa2a66f42f3a5a755f5f4adc858
I20260709 12:59:39.103786  8908 tablet_copy_source_session.cc:215] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.107702  9666 tablet_copy_client.cc:806] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.108290  9663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f192c6b771043a893d6fd420c9b9e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.108349  9666 tablet_copy_client.cc:670] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.118937  9666 tablet_copy_client.cc:538] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.120990  8834 tablet_copy_source_session.cc:215] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.124593  9668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 234e3fa2a66f42f3a5a755f5f4adc858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.125708  9663 tablet_copy_client.cc:806] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.126287  9663 tablet_copy_client.cc:670] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.130899  9666 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:39.133069  9663 tablet_copy_client.cc:538] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.143741  9668 tablet_copy_client.cc:806] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.144409  9668 tablet_copy_client.cc:670] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.148178  9668 tablet_copy_client.cc:538] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.149442  9663 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:39.181672  9668 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:39.190388  9666 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.191753  9666 tablet_bootstrap.cc:492] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:39.192736  9666 ts_tablet_manager.cc:1403] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.062s	user 0.022s	sys 0.013s
I20260709 12:59:39.201102  9666 raft_consensus.cc:359] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.201967  9666 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:39.202667  9666 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.205428  9666 ts_tablet_manager.cc:1434] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:39.207737  9668 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.207883  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-33a0adf8736244d3b208cbc85700f0f9 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:39.208350  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-33a0adf8736244d3b208cbc85700f0f9 on tablet 33a0adf8736244d3b208cbc85700f0f9 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:39.208472  9663 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.208937  9668 tablet_bootstrap.cc:492] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:39.209419  9663 tablet_bootstrap.cc:492] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:39.209672  9668 ts_tablet_manager.cc:1403] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.008s
I20260709 12:59:39.210076  9663 ts_tablet_manager.cc:1403] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.012s
I20260709 12:59:39.212894  9663 raft_consensus.cc:359] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.213883  9663 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:39.212956  9668 raft_consensus.cc:359] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.215605  9668 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:39.217142  9668 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.215626  9663 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:39.219986  9663 ts_tablet_manager.cc:1434] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:59:39.220533  9668 ts_tablet_manager.cc:1434] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:59:39.222101  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-234e3fa2a66f42f3a5a755f5f4adc858 received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:39.222514  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-234e3fa2a66f42f3a5a755f5f4adc858 on tablet 234e3fa2a66f42f3a5a755f5f4adc858 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:39.223671  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-3f192c6b771043a893d6fd420c9b9e7d received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:39.224063  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-3f192c6b771043a893d6fd420c9b9e7d on tablet 3f192c6b771043a893d6fd420c9b9e7d with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:39.414059  9447 raft_consensus.cc:1217] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.494922  9447 raft_consensus.cc:1217] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.512885  9447 raft_consensus.cc:1217] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.545758  9447 raft_consensus.cc:1217] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.888223  9639 raft_consensus.cc:1064] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:39.890674  9639 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:39.902107  8822 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.904258  9447 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.904274  8741 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:39.904376  9639 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:59:39.906656  9645 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:39.907424  9675 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:39.931849  9675 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:39.941016  9447 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:39.941928  8741 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:39.956048  8647 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:39.967104  8820 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:39.993263  9637 raft_consensus.cc:1064] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:39.995672  9637 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.007076  9447 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.008020  8896 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.008570  9686 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.010154  8741 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.010936  9676 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.023651  9686 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.029829  8892 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.055780  9449 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.057773  9645 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.061120  8749 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.063900  9645 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.066156  9686 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.068372  9639 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.073060  8741 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.079851  9449 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.082031  9448 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.083525  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3f192c6b771043a893d6fd420c9b9e7d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.084288  8749 raft_consensus.cc:2955] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.097267  9584 raft_consensus.cc:1064] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:40.096930  8646 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:40.099613  9584 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.106400  8896 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.108903  9675 raft_consensus.cc:1064] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:40.111263  9675 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.118415  8643 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:40.125522  9449 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.126896  9584 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.127825  8892 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.128419  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f192c6b771043a893d6fd420c9b9e7d with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:40.133165  9695 tablet_replica.cc:333] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:40.133212  9584 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:40.134879  9695 raft_consensus.cc:2243] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.137225  9695 raft_consensus.cc:2272] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.140594  8820 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.148429  8820 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.149391  9584 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.151531  9682 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.159153  9448 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.159904  8820 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.166047  9682 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.167992  9695 ts_tablet_manager.cc:1916] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.183815  9448 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.183995  8741 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.185446  9689 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.194073  9686 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.195712  9447 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.208900  8647 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.210848  8895 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.211071  9651 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.214004  9689 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.215408  9695 ts_tablet_manager.cc:1929] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.215817  9695 log.cc:1199] T 3f192c6b771043a893d6fd420c9b9e7d P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/3f192c6b771043a893d6fd420c9b9e7d
I20260709 12:59:40.220460  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 3f192c6b771043a893d6fd420c9b9e7d (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:40.221583  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 234e3fa2a66f42f3a5a755f5f4adc858 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:40.230916  9645 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.234064  8895 raft_consensus.cc:2955] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.237622  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 234e3fa2a66f42f3a5a755f5f4adc858 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.241951  8892 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.249205  9448 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.267951  9675 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.275851  8823 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.280918  8644 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.281738  9449 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.296226  8647 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.305840  8823 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.317924  9701 tablet_replica.cc:333] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:40.319887  9701 raft_consensus.cc:2243] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.320796  9701 raft_consensus.cc:2272] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.336858  9701 ts_tablet_manager.cc:1916] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.343348  8741 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.347244  8892 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:40.354811  9449 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.356714  9582 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.358867  8823 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.360257  9682 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.365084  9651 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.368080  9449 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.367928  8820 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.370627  9675 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.378909  9447 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.383553  9639 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.404116  9448 raft_consensus.cc:2955] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.409078  9701 ts_tablet_manager.cc:1929] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.409593  9701 log.cc:1199] T 234e3fa2a66f42f3a5a755f5f4adc858 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/234e3fa2a66f42f3a5a755f5f4adc858
I20260709 12:59:40.417042  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 33a0adf8736244d3b208cbc85700f0f9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.419768  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 234e3fa2a66f42f3a5a755f5f4adc858 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:40.420460  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 029b57162d72422bba468c7d86b67fca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.428440  8644 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:40.430868  8820 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.452446  8646 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:40.454999  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 029b57162d72422bba468c7d86b67fca with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:40.468576  9704 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.470767  9706 tablet_replica.cc:333] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:40.474020  9706 raft_consensus.cc:2243] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.473958  8822 raft_consensus.cc:2955] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:40.475342  9706 raft_consensus.cc:2272] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.477906  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 33a0adf8736244d3b208cbc85700f0f9 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:40.478788  9701 tablet_replica.cc:333] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:40.479808  9701 raft_consensus.cc:2243] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.480943  9701 pending_rounds.cc:70] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 1 pending ops.
I20260709 12:59:40.481959  9706 ts_tablet_manager.cc:1916] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.481244  9701 pending_rounds.cc:77] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305633710530322432 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "33a0adf8736244d3b208cbc85700f0f9" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } } }
I20260709 12:59:40.482625  9701 raft_consensus.cc:2917] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:59:40.483067  9701 raft_consensus.cc:2889] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:59:40.483419  9701 raft_consensus.cc:2272] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.486943  9701 ts_tablet_manager.cc:1916] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:40.502539  9706 ts_tablet_manager.cc:1929] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.502991  9706 log.cc:1199] T 029b57162d72422bba468c7d86b67fca P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/029b57162d72422bba468c7d86b67fca
I20260709 12:59:40.503403  9701 ts_tablet_manager.cc:1929] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:40.503875  9701 log.cc:1199] T 33a0adf8736244d3b208cbc85700f0f9 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/33a0adf8736244d3b208cbc85700f0f9
I20260709 12:59:40.504509  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 029b57162d72422bba468c7d86b67fca (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:40.505549  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 33a0adf8736244d3b208cbc85700f0f9 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:41.549088  8820 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:41.556188  8788 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0192ad465a724ab581b9529345f73bba. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.556598  8897 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.558246  9686 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.558519  8741 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.560112  9676 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.566790  9686 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.569200  8896 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.569622  8741 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.585085  8643 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.593065  8820 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:41.599498  8896 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:41.600373  8788 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fa55bf932444ccebb2b7f6583ead84f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.601572  8741 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.601333  9686 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:41.603360  9676 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.620630  9711 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.622752  8741 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.622902  8896 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.640489  8741 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:41.643361  8644 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.652067  8896 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.653164  8820 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.655241  9582 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:59:41.660207  9582 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:41.667431  8713 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc6dcf2342014ed6b65dd5af29d18d3b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.669422  9702 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.674062  8897 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.687417  8823 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:41.698671  8822 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.698658  8643 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.738839  8897 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:41.739426  8741 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:41.739611  8788 consensus_peers.cc:597] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 932d7bc525ba4af99f7d6b52fc1d7337. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.740175  9676 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.741586  9711 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:41.756633  9708 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.758211  8741 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.768085  8896 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:41.776753  8643 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:42.008327  9727 ts_tablet_manager.cc:933] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:42.017652  9727 tablet_copy_client.cc:323] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:42.019477  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 6fa55bf932444ccebb2b7f6583ead84f from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:42.020001  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 6fa55bf932444ccebb2b7f6583ead84f from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-6fa55bf932444ccebb2b7f6583ead84f
I20260709 12:59:42.029682  8834 tablet_copy_source_session.cc:215] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.035938  9727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa55bf932444ccebb2b7f6583ead84f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.051836  9727 tablet_copy_client.cc:806] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.052443  9727 tablet_copy_client.cc:670] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.056468  9727 tablet_copy_client.cc:538] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.065145  9727 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:42.085469  9727 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.086385  9727 tablet_bootstrap.cc:492] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:42.087024  9727 ts_tablet_manager.cc:1403] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.022s	user 0.022s	sys 0.001s
I20260709 12:59:42.089906  9727 raft_consensus.cc:359] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.090728  9727 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:42.091368  9727 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.094504  9727 ts_tablet_manager.cc:1434] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:59:42.096092  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-6fa55bf932444ccebb2b7f6583ead84f received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:42.096451  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-6fa55bf932444ccebb2b7f6583ead84f on tablet 6fa55bf932444ccebb2b7f6583ead84f with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:42.113893  9727 ts_tablet_manager.cc:933] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:42.116886  9727 tablet_copy_client.cc:323] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:42.118499  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 0192ad465a724ab581b9529345f73bba from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:42.119042  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 0192ad465a724ab581b9529345f73bba from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-0192ad465a724ab581b9529345f73bba
I20260709 12:59:42.124635  8834 tablet_copy_source_session.cc:215] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.127360  9727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ad465a724ab581b9529345f73bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.140190  9727 tablet_copy_client.cc:806] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.140813  9727 tablet_copy_client.cc:670] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.144059  9727 tablet_copy_client.cc:538] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.150017  9727 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:42.158068  9732 ts_tablet_manager.cc:933] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:42.166464  9734 ts_tablet_manager.cc:933] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:42.181942  9732 tablet_copy_client.cc:323] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:42.182862  9734 tablet_copy_client.cc:323] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:42.183739  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 932d7bc525ba4af99f7d6b52fc1d7337 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:42.184290  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 932d7bc525ba4af99f7d6b52fc1d7337 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-932d7bc525ba4af99f7d6b52fc1d7337
I20260709 12:59:42.184878  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet dc6dcf2342014ed6b65dd5af29d18d3b from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:42.185452  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet dc6dcf2342014ed6b65dd5af29d18d3b from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-dc6dcf2342014ed6b65dd5af29d18d3b
I20260709 12:59:42.189210  9727 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.190171  9727 tablet_bootstrap.cc:492] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:42.190891  9727 ts_tablet_manager.cc:1403] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.007s
I20260709 12:59:42.194070  8834 tablet_copy_source_session.cc:215] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.193923  8759 tablet_copy_source_session.cc:215] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.194842  9727 raft_consensus.cc:359] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.195850  9727 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:42.196565  9727 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.199280  9727 ts_tablet_manager.cc:1434] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:42.202414  9734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6dcf2342014ed6b65dd5af29d18d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.201188  8833 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-0192ad465a724ab581b9529345f73bba received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:42.203895  8833 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-0192ad465a724ab581b9529345f73bba on tablet 0192ad465a724ab581b9529345f73bba with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:42.214725  9732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932d7bc525ba4af99f7d6b52fc1d7337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.224715  9734 tablet_copy_client.cc:806] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.225353  9734 tablet_copy_client.cc:670] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.230769  9734 tablet_copy_client.cc:538] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.231149  9732 tablet_copy_client.cc:806] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.231711  9732 tablet_copy_client.cc:670] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.235224  9732 tablet_copy_client.cc:538] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.237824  9734 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:42.247236  9732 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:42.264766  9734 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.265689  9734 tablet_bootstrap.cc:492] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:42.266686  9734 ts_tablet_manager.cc:1403] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.019s
I20260709 12:59:42.269562  9734 raft_consensus.cc:359] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.270433  9734 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:42.271106  9734 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.275713  9734 ts_tablet_manager.cc:1434] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:59:42.289023  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-dc6dcf2342014ed6b65dd5af29d18d3b received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:42.289485  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-dc6dcf2342014ed6b65dd5af29d18d3b on tablet dc6dcf2342014ed6b65dd5af29d18d3b with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:42.306129  9732 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.307160  9732 tablet_bootstrap.cc:492] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:42.307838  9732 ts_tablet_manager.cc:1403] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.005s
I20260709 12:59:42.311093  9732 raft_consensus.cc:359] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.313472  9732 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:42.314172  9732 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:42.320222  9732 ts_tablet_manager.cc:1434] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:59:42.321869  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-932d7bc525ba4af99f7d6b52fc1d7337 received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:42.322286  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-932d7bc525ba4af99f7d6b52fc1d7337 on tablet 932d7bc525ba4af99f7d6b52fc1d7337 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:42.470150  9448 raft_consensus.cc:1217] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:42.561913  9448 raft_consensus.cc:1217] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:42.599773  9448 raft_consensus.cc:1217] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:42.634835  9448 raft_consensus.cc:1217] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:42.978199  9739 raft_consensus.cc:1064] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:42.980826  9739 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:42.990213  9676 raft_consensus.cc:1064] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:42.992498  8822 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:42.992194  9676 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:42.993992  8895 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:42.994019  9739 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:42.994452  9448 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:42.995463  9723 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:42.997320  9720 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.013821  9447 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.013329  9723 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.027429  8892 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.031601  8643 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.032414  8741 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.037951  9708 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.040797  9448 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.044579  9676 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.047243  9686 raft_consensus.cc:1064] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:43.050689  9686 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:43.133908  9449 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.139062  9676 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:59:43.155144  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 120206us
I20260709 12:59:43.200052  9676 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.220878  8740 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.239465  8822 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.246176  9751 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.260741  8895 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.268121  8898 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 12:59:43.337646  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 55760us
W20260709 12:59:43.339079  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77078.9us
W20260709 12:59:43.386744  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 93692.5us
W20260709 12:59:43.393352  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 95426.2us
I20260709 12:59:43.406767  9708 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:59:43.421862  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 94744.5us
W20260709 12:59:43.493799  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 108055us
I20260709 12:59:43.480201  9767 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.506587  9756 raft_consensus.cc:1064] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:43.521489  9676 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.527227  8898 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.555168  8744 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.626701  9756 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:43.643244  8740 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.663710  9448 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.667016  9449 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.711392  8894 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.717681  8646 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.720023  8646 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.765842  9447 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.769507  9676 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.784941  8740 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.787881  8898 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.789966  9676 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.803390  8744 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:43.802387  8823 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:43.808910  9447 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.808535  8819 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:43.811530  9723 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.819752  9764 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.821571  8744 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.823115  9676 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.825984  9447 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.827473  9676 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.830157  9756 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.839092  8744 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.842492  8895 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.850764  9776 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.867550  8647 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.868592  9449 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.835947  9447 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.871320  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6fa55bf932444ccebb2b7f6583ead84f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.870829  9446 raft_consensus.cc:2955] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.879050  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fa55bf932444ccebb2b7f6583ead84f with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:43.887666  9756 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:43.891165  9786 tablet_replica.cc:333] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:43.895172  9779 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.894112  9786 raft_consensus.cc:2243] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:43.897547  9786 raft_consensus.cc:2272] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:43.902390  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc6dcf2342014ed6b65dd5af29d18d3b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.912146  9786 ts_tablet_manager.cc:1916] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:43.939072  8644 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.938031  9447 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.945358  9582 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.947286  8895 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.961468  9686 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.970731  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet dc6dcf2342014ed6b65dd5af29d18d3b with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:43.988348  8897 raft_consensus.cc:2955] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:43.995407  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0192ad465a724ab581b9529345f73bba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.998135  8646 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.010617  8898 raft_consensus.cc:2955] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.028204  9686 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.029992  9448 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.045832  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 0192ad465a724ab581b9529345f73bba with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:44.059687  8898 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.065860  9788 tablet_replica.cc:333] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:44.071861  9788 raft_consensus.cc:2243] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.071278  8740 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.073014  9789 tablet_replica.cc:333] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:44.074584  9789 raft_consensus.cc:2243] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.086650  9789 raft_consensus.cc:2272] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.073540  9788 raft_consensus.cc:2272] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.079110  8643 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.101501  9788 ts_tablet_manager.cc:1916] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.103780  9786 ts_tablet_manager.cc:1929] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.104089  9789 ts_tablet_manager.cc:1916] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.104281  9786 log.cc:1199] T 6fa55bf932444ccebb2b7f6583ead84f P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/6fa55bf932444ccebb2b7f6583ead84f
I20260709 12:59:44.105875  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 6fa55bf932444ccebb2b7f6583ead84f (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:44.114401  8820 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:44.120550  9446 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.122327  8898 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.123397  9686 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.125826  9686 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.127439  9789 ts_tablet_manager.cc:1929] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.127877  9789 log.cc:1199] T 0192ad465a724ab581b9529345f73bba P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/0192ad465a724ab581b9529345f73bba
I20260709 12:59:44.130465  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 0192ad465a724ab581b9529345f73bba (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:44.134974  9446 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.133473  9686 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.144230  8897 raft_consensus.cc:2955] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:44.148293  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 932d7bc525ba4af99f7d6b52fc1d7337 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:44.156847  8646 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:44.170318  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 932d7bc525ba4af99f7d6b52fc1d7337 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:44.170948  9788 ts_tablet_manager.cc:1929] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.171378  9788 log.cc:1199] T dc6dcf2342014ed6b65dd5af29d18d3b P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/dc6dcf2342014ed6b65dd5af29d18d3b
I20260709 12:59:44.171416  9789 tablet_replica.cc:333] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:44.173031  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet dc6dcf2342014ed6b65dd5af29d18d3b (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:44.174057  9789 raft_consensus.cc:2243] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.178918  9789 raft_consensus.cc:2272] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.183563  9789 ts_tablet_manager.cc:1916] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.194742  9789 ts_tablet_manager.cc:1929] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.195088  9789 log.cc:1199] T 932d7bc525ba4af99f7d6b52fc1d7337 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/932d7bc525ba4af99f7d6b52fc1d7337
I20260709 12:59:44.196497  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 932d7bc525ba4af99f7d6b52fc1d7337 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:45.242625  8740 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:45.248754  8713 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a07b06e72d474091a0bcf7de181b509f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.249943  8895 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.250530  8823 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.251757  9723 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.252609  9582 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.259111  9723 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.262295  8823 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.264708  8895 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.279642  8823 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.292861  8646 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.327548  8897 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.329178  9756 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:45.332700  8788 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 265e5f9784a6426482f5948fbfcc995c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.335433  8749 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.337718  9776 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.369686  9756 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.370632  8897 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.373692  8740 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.389279  8823 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.398806  8644 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 12:59:45.401559  8788 consensus_peers.cc:597] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 410ca1c9d2fd42c68bdf6bb996508fd9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.403872  8740 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.405483  8897 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.406634  9801 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.414552  9776 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:45.425215  9756 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.429198  8898 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.439306  8819 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.436964  8740 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.447654  8644 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.450263  8749 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:45.450834  8788 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3006e1c3fb5b4c66852cbc955265f1cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.454412  8898 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.455469  9756 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.457216  9801 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.467403  9756 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.469272  8740 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.471266  8898 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.488310  8647 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:45.745455  9809 ts_tablet_manager.cc:933] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:45.747323  9809 tablet_copy_client.cc:323] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:45.748711  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet a07b06e72d474091a0bcf7de181b509f from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:45.749251  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet a07b06e72d474091a0bcf7de181b509f from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-a07b06e72d474091a0bcf7de181b509f
I20260709 12:59:45.756801  8759 tablet_copy_source_session.cc:215] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:45.759907  9809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a07b06e72d474091a0bcf7de181b509f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:45.776767  9809 tablet_copy_client.cc:806] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:45.777345  9809 tablet_copy_client.cc:670] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:45.780912  9809 tablet_copy_client.cc:538] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:45.788638  9809 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:45.808399  9809 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.809314  9809 tablet_bootstrap.cc:492] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:45.810002  9809 ts_tablet_manager.cc:1403] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.004s
I20260709 12:59:45.812197  9809 raft_consensus.cc:359] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.812848  9809 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:45.813381  9809 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.817366  9809 ts_tablet_manager.cc:1434] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:45.819370  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-a07b06e72d474091a0bcf7de181b509f received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:45.819761  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-a07b06e72d474091a0bcf7de181b509f on tablet a07b06e72d474091a0bcf7de181b509f with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:45.874876  9809 ts_tablet_manager.cc:933] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:45.876470  9809 tablet_copy_client.cc:323] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:45.877970  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:45.878422  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 410ca1c9d2fd42c68bdf6bb996508fd9 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-410ca1c9d2fd42c68bdf6bb996508fd9
I20260709 12:59:45.884883  8834 tablet_copy_source_session.cc:215] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:45.887917  9809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 410ca1c9d2fd42c68bdf6bb996508fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:45.903883  9809 tablet_copy_client.cc:806] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:45.904464  9809 tablet_copy_client.cc:670] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:45.907869  9809 tablet_copy_client.cc:538] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:45.915129  9809 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:45.937630  9814 ts_tablet_manager.cc:933] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:45.940277  9814 tablet_copy_client.cc:323] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:45.941730  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 265e5f9784a6426482f5948fbfcc995c from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:45.942240  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 265e5f9784a6426482f5948fbfcc995c from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-265e5f9784a6426482f5948fbfcc995c
I20260709 12:59:45.947940  9809 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.948807  9809 tablet_bootstrap.cc:492] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:45.949460  9809 ts_tablet_manager.cc:1403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.035s	user 0.027s	sys 0.004s
I20260709 12:59:45.949849  8834 tablet_copy_source_session.cc:215] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:45.952886  9814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 265e5f9784a6426482f5948fbfcc995c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:45.952937  9809 raft_consensus.cc:359] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.953857  9809 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:45.954497  9809 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:45.956797  9809 ts_tablet_manager.cc:1434] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 12:59:45.965464  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-410ca1c9d2fd42c68bdf6bb996508fd9 received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:45.965876  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-410ca1c9d2fd42c68bdf6bb996508fd9 on tablet 410ca1c9d2fd42c68bdf6bb996508fd9 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:45.966679  9814 tablet_copy_client.cc:806] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:45.967413  9814 tablet_copy_client.cc:670] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:45.970582  9814 tablet_copy_client.cc:538] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:45.979697  9814 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:45.999022  9814 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.999876  9814 tablet_bootstrap.cc:492] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:46.000506  9814 ts_tablet_manager.cc:1403] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:59:46.003321  9814 raft_consensus.cc:359] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:46.004128  9814 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:46.004738  9814 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:46.007071  9814 ts_tablet_manager.cc:1434] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:59:46.010501  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-265e5f9784a6426482f5948fbfcc995c received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:46.010867  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-265e5f9784a6426482f5948fbfcc995c on tablet 265e5f9784a6426482f5948fbfcc995c with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:46.015455  9814 ts_tablet_manager.cc:933] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:46.021210  9814 tablet_copy_client.cc:323] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:46.022676  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 3006e1c3fb5b4c66852cbc955265f1cb from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:46.023173  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 3006e1c3fb5b4c66852cbc955265f1cb from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-3006e1c3fb5b4c66852cbc955265f1cb
I20260709 12:59:46.030514  8834 tablet_copy_source_session.cc:215] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.033632  9814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3006e1c3fb5b4c66852cbc955265f1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.049036  9814 tablet_copy_client.cc:806] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.049593  9814 tablet_copy_client.cc:670] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.055685  9814 tablet_copy_client.cc:538] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.063709  9814 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:46.082768  9814 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.083592  9814 tablet_bootstrap.cc:492] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:46.084208  9814 ts_tablet_manager.cc:1403] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:59:46.086951  9814 raft_consensus.cc:359] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:46.087751  9814 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:46.088358  9814 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:46.090770  9814 ts_tablet_manager.cc:1434] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:46.092365  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-3006e1c3fb5b4c66852cbc955265f1cb received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:46.092759  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-3006e1c3fb5b4c66852cbc955265f1cb on tablet 3006e1c3fb5b4c66852cbc955265f1cb with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:46.245405  9446 raft_consensus.cc:1217] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.429232  9446 raft_consensus.cc:1217] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.527572  9446 raft_consensus.cc:1217] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.539840  9448 raft_consensus.cc:1217] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.670735  9582 raft_consensus.cc:1064] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:46.676396  9582 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:46.684079  9446 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.686012  8819 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.689018  8895 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.689345  9828 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.691538  9723 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.692270  9828 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.757895  9829 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.760710  9446 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.764752  8895 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.776225  8646 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.780052  8820 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.819902  8740 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:46.828449  8895 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:46.829828  9446 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:46.829844  9828 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:46.836697  9828 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:46.841741  9582 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.846639  8895 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.848630  9446 raft_consensus.cc:2955] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.855099  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a07b06e72d474091a0bcf7de181b509f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:46.863466  8647 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.877673  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet a07b06e72d474091a0bcf7de181b509f with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:46.881625  9837 tablet_replica.cc:333] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:46.883540  9837 raft_consensus.cc:2243] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:46.884229  9837 raft_consensus.cc:2272] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:46.887816  9837 ts_tablet_manager.cc:1916] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:46.904265  9837 ts_tablet_manager.cc:1929] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:46.904783  9837 log.cc:1199] T a07b06e72d474091a0bcf7de181b509f P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/a07b06e72d474091a0bcf7de181b509f
I20260709 12:59:46.904999  9824 raft_consensus.cc:1064] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:46.907115  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet a07b06e72d474091a0bcf7de181b509f (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:46.907778  9824 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:46.916258  8895 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.920637  8740 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.919852  9446 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.921137  9756 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.922814  9801 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.923606  9824 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.948486  8895 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.948694  9801 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.954056  8740 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.966365  8647 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.976384  9447 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:46.996595  8820 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:47.001812  9447 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.003235  9756 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.003327  8740 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.004735  9825 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.009022  9756 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.010660  9446 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.018136  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 265e5f9784a6426482f5948fbfcc995c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.027335  8740 raft_consensus.cc:2955] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.032967  8644 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.056743  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 265e5f9784a6426482f5948fbfcc995c with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:47.060843  9842 tablet_replica.cc:333] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:47.062204  9842 raft_consensus.cc:2243] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.063045  9842 raft_consensus.cc:2272] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.067293  9842 ts_tablet_manager.cc:1916] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.082870  9842 ts_tablet_manager.cc:1929] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.083278  9842 log.cc:1199] T 265e5f9784a6426482f5948fbfcc995c P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/265e5f9784a6426482f5948fbfcc995c
I20260709 12:59:47.085144  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 265e5f9784a6426482f5948fbfcc995c (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:47.104035  9756 raft_consensus.cc:1064] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:47.106739  9756 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:47.113052  9446 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.113792  8740 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.114279  8895 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.114933  9824 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.115929  9776 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.116699  9825 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.138370  9776 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.143676  9822 raft_consensus.cc:1064] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:47.143108  8740 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.146355  9822 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:47.154166  9447 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.161469  8897 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.164829  8647 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.182510  8823 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:47.189352  8898 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.196807  8740 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.200011  9446 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.200625  8898 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.205401  9801 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.206745  9448 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.209522  9823 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.211313  9823 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.212709  9825 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.215040  9825 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.229346  9756 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.231024  8895 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.232405  9446 raft_consensus.cc:2955] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.257913  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.257556  9822 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.261420  8740 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.276249  9446 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.279902  8646 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.281984  8644 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.291427  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 410ca1c9d2fd42c68bdf6bb996508fd9 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:47.279102  8898 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.316895  9851 tablet_replica.cc:333] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:47.331559  9851 raft_consensus.cc:2243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.331210  8820 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:47.332329  9851 raft_consensus.cc:2272] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.335917  9851 ts_tablet_manager.cc:1916] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.340183  9448 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.341959  9756 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.342628  8740 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.345618  9822 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.349745  9776 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.351696  9448 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.357851  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3006e1c3fb5b4c66852cbc955265f1cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.370687  8740 raft_consensus.cc:2955] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.371836  8647 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:47.384924  9851 ts_tablet_manager.cc:1929] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.385468  9851 log.cc:1199] T 410ca1c9d2fd42c68bdf6bb996508fd9 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/410ca1c9d2fd42c68bdf6bb996508fd9
I20260709 12:59:47.387331  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 410ca1c9d2fd42c68bdf6bb996508fd9 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:47.390915  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 3006e1c3fb5b4c66852cbc955265f1cb with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:47.392040  9842 tablet_replica.cc:333] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:47.393054  9842 raft_consensus.cc:2243] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.394356  9842 raft_consensus.cc:2272] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.398237  9842 ts_tablet_manager.cc:1916] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.411255  9842 ts_tablet_manager.cc:1929] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.411630  9842 log.cc:1199] T 3006e1c3fb5b4c66852cbc955265f1cb P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/3006e1c3fb5b4c66852cbc955265f1cb
I20260709 12:59:47.412869  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 3006e1c3fb5b4c66852cbc955265f1cb (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:48.467103  8895 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:48.473179  8820 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:48.474504  8862 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd126387f2fb45f98d7ca0807566c688. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.474716  9773 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.475455  8740 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.476538  9773 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.481936  9820 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.483374  8820 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.484225  8740 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.496889  8897 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:48.498605  8647 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.509063  8820 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.511150  9858 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:48.513979  8862 consensus_peers.cc:597] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f45eaa0e72045d5ae7ba8f95ca2ce75. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.514444  8740 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.515622  9773 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:48.525856  9858 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.527436  8740 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.529023  8820 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.554764  8644 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.551587  8740 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:48.580331  8895 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:48.581001  8713 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7884996e8b84433f80124e36111f58ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.583650  8820 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.582718  9828 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.585601  9829 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.601986  9852 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.603716  8895 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.606146  8820 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.619911  8895 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:48.624347  8647 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.636891  8820 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.638666  9773 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:48.639703  8862 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9402cffd3f284d28bf964bd56144a185. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.640617  8749 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.642242  9773 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.648532  9858 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.651834  8744 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.658601  8823 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.672134  8647 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:48.949177  9870 ts_tablet_manager.cc:933] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:48.957587  9870 tablet_copy_client.cc:323] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:48.959358  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet fd126387f2fb45f98d7ca0807566c688 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:48.959854  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet fd126387f2fb45f98d7ca0807566c688 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-fd126387f2fb45f98d7ca0807566c688
I20260709 12:59:48.968330  8908 tablet_copy_source_session.cc:215] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:48.971310  9872 ts_tablet_manager.cc:933] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:48.973620  9870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd126387f2fb45f98d7ca0807566c688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:48.977371  9872 tablet_copy_client.cc:323] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:48.979070  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:48.979568  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-1f45eaa0e72045d5ae7ba8f95ca2ce75
I20260709 12:59:48.986831  8908 tablet_copy_source_session.cc:215] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:48.991062  9872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:48.993260  9870 tablet_copy_client.cc:806] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:48.993899  9870 tablet_copy_client.cc:670] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:48.998687  9870 tablet_copy_client.cc:538] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.006208  9872 tablet_copy_client.cc:806] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.006655  9870 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:49.006856  9872 tablet_copy_client.cc:670] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.010741  9872 tablet_copy_client.cc:538] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.019953  9872 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:49.046186  9870 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.047542  9870 tablet_bootstrap.cc:492] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:49.048527  9870 ts_tablet_manager.cc:1403] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.011s
I20260709 12:59:49.053984  9870 raft_consensus.cc:359] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.054834  9870 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:49.055048  9874 ts_tablet_manager.cc:933] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:49.056166  9872 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.055560  9870 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.057116  9872 tablet_bootstrap.cc:492] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:49.057888  9872 ts_tablet_manager.cc:1403] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.004s
I20260709 12:59:49.060716  9872 raft_consensus.cc:359] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.061550  9872 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:49.062196  9872 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.065188  9874 tablet_copy_client.cc:323] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:49.068202  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 7884996e8b84433f80124e36111f58ac from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:49.068717  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 7884996e8b84433f80124e36111f58ac from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-7884996e8b84433f80124e36111f58ac
I20260709 12:59:49.071280  9870 ts_tablet_manager.cc:1434] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.005s
I20260709 12:59:49.073747  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-fd126387f2fb45f98d7ca0807566c688 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:49.074201  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-fd126387f2fb45f98d7ca0807566c688 on tablet fd126387f2fb45f98d7ca0807566c688 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:49.080905  8759 tablet_copy_source_session.cc:215] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.082077  9872 ts_tablet_manager.cc:1434] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260709 12:59:49.085798  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-1f45eaa0e72045d5ae7ba8f95ca2ce75 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:49.086199  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-1f45eaa0e72045d5ae7ba8f95ca2ce75 on tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:49.086124  9874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7884996e8b84433f80124e36111f58ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.103214  9874 tablet_copy_client.cc:806] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.103816  9874 tablet_copy_client.cc:670] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.114084  9874 tablet_copy_client.cc:538] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.126579  9874 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:49.146503  9874 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.148129  9874 tablet_bootstrap.cc:492] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:49.148744  9874 ts_tablet_manager.cc:1403] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.022s	user 0.021s	sys 0.000s
I20260709 12:59:49.151612  9874 raft_consensus.cc:359] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.152176  9874 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:49.152742  9874 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.157347  9874 ts_tablet_manager.cc:1434] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:59:49.158808  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-7884996e8b84433f80124e36111f58ac received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:49.159222  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-7884996e8b84433f80124e36111f58ac on tablet 7884996e8b84433f80124e36111f58ac with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:49.169771  9874 ts_tablet_manager.cc:933] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:49.171677  9874 tablet_copy_client.cc:323] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:49.173251  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 9402cffd3f284d28bf964bd56144a185 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:49.173759  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 9402cffd3f284d28bf964bd56144a185 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-9402cffd3f284d28bf964bd56144a185
I20260709 12:59:49.181975  8908 tablet_copy_source_session.cc:215] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.184437  9874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9402cffd3f284d28bf964bd56144a185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.197263  9874 tablet_copy_client.cc:806] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.197835  9874 tablet_copy_client.cc:670] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.201388  9874 tablet_copy_client.cc:538] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.210145  9874 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:49.242648  9874 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.243501  9874 tablet_bootstrap.cc:492] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:49.244138  9874 ts_tablet_manager.cc:1403] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.011s
I20260709 12:59:49.247020  9874 raft_consensus.cc:359] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.247821  9874 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:49.248420  9874 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:49.251132  9874 ts_tablet_manager.cc:1434] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:49.252776  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-9402cffd3f284d28bf964bd56144a185 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:49.253160  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-9402cffd3f284d28bf964bd56144a185 on tablet 9402cffd3f284d28bf964bd56144a185 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:49.397980  9448 raft_consensus.cc:1217] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:49.447474  9448 raft_consensus.cc:1217] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:49.523945  9448 raft_consensus.cc:1217] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:49.652072  9448 raft_consensus.cc:1217] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:49.955559  9885 raft_consensus.cc:1064] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:49.958133  9885 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:49.964398  8820 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:49.965797  8740 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:49.965794  9885 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:49.967103  9448 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:49.968727  9885 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:49.971382  9885 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:49.985329  9820 raft_consensus.cc:1064] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:49.988044  9820 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:49.989084  9883 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:49.991919  8744 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:49.996490  9448 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.007089  9829 raft_consensus.cc:1064] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:50.012665  9829 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.017566  8820 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.024070  8646 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.039219  8897 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.039721  9446 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.041488  9852 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.045255  8820 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.046509  8823 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.046751  9885 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.051003  9447 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.053421  8744 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.054261  9886 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.063261  9886 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.070034  9891 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.072623  9891 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.082221  9773 raft_consensus.cc:1064] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:50.084980  9773 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.100466  9891 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.115036  8740 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.115998  8897 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.121506  9446 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.112396  9852 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.133749  9447 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.220880  8820 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.222405  8744 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.226681  9820 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.234501  9446 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.237612  9820 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:59:50.215086  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53647.4us
I20260709 12:59:50.245232  8818 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.244786  8823 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.253355  9893 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.285861  9892 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.293712  8820 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.301609  8897 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.306535  8740 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.308691  8749 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.311465  9893 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.318806  9820 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.321890  9446 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.326280  8740 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.328708  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fd126387f2fb45f98d7ca0807566c688 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.314879  8647 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.336373  8647 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.342172  9446 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.343474  9820 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.347797  9446 raft_consensus.cc:2955] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.366051  8643 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.367323  8643 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.384601  8740 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.395808  9446 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.397698  9911 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.398151  8820 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.400040  9900 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.401145  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet fd126387f2fb45f98d7ca0807566c688 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:50.407179  9900 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.419132  8820 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.423555  9448 raft_consensus.cc:2955] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.430305  8897 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.430636  8898 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.433759  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7884996e8b84433f80124e36111f58ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.437680  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:50.440333  9446 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.441712  9820 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.443727  8740 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.445082  9820 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.451170  9885 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.452081  8644 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.453430  8740 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.454288  9447 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.455884  9910 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:50.454262  9446 raft_consensus.cc:2955] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.459573  8823 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.459995  9918 tablet_replica.cc:333] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:50.462495  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9402cffd3f284d28bf964bd56144a185 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.460906  9820 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.462393  9918 raft_consensus.cc:2243] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.470969  9918 raft_consensus.cc:2272] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.477864  9918 ts_tablet_manager.cc:1916] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.476833  9885 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.480017  8819 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.502962  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 7884996e8b84433f80124e36111f58ac with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:50.482093  9448 raft_consensus.cc:2955] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.482156  8646 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.518038  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.525408  9919 tablet_replica.cc:333] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:50.533450  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 9402cffd3f284d28bf964bd56144a185 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:50.538327  9919 raft_consensus.cc:2243] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.539121  9919 raft_consensus.cc:2272] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.545462  9918 ts_tablet_manager.cc:1929] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.549000  9918 log.cc:1199] T fd126387f2fb45f98d7ca0807566c688 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/fd126387f2fb45f98d7ca0807566c688
I20260709 12:59:50.541117  8647 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:50.550729  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet fd126387f2fb45f98d7ca0807566c688 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:50.550882  9918 tablet_replica.cc:333] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:50.547319  9919 ts_tablet_manager.cc:1916] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.552042  9918 raft_consensus.cc:2243] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.555346  9918 raft_consensus.cc:2272] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.567184  9918 ts_tablet_manager.cc:1916] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.568598  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:50.570365  9920 tablet_replica.cc:333] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:50.571381  9920 raft_consensus.cc:2243] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.572031  9920 raft_consensus.cc:2272] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.575330  9920 ts_tablet_manager.cc:1916] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.582305  9918 ts_tablet_manager.cc:1929] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.582852  9918 log.cc:1199] T 9402cffd3f284d28bf964bd56144a185 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/9402cffd3f284d28bf964bd56144a185
I20260709 12:59:50.584462  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 9402cffd3f284d28bf964bd56144a185 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:50.584589  9919 ts_tablet_manager.cc:1929] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.585016  9919 log.cc:1199] T 7884996e8b84433f80124e36111f58ac P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/7884996e8b84433f80124e36111f58ac
I20260709 12:59:50.586629  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 7884996e8b84433f80124e36111f58ac (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:50.593672  9920 ts_tablet_manager.cc:1929] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.594442  9920 log.cc:1199] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/1f45eaa0e72045d5ae7ba8f95ca2ce75
I20260709 12:59:50.598088  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 1f45eaa0e72045d5ae7ba8f95ca2ce75 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:50.622906  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:50.623636  8820 raft_consensus.cc:606] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:50.625798  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:50.626222  8820 raft_consensus.cc:606] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:50.637276  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44b3244bf12f42aba79421575a96c621"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:50.637832  8749 raft_consensus.cc:606] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:50.640091  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:50.640528  8820 raft_consensus.cc:606] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:50.642923  8823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:50.643450  8823 raft_consensus.cc:606] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:50.644523  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:50.645581  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:50.694427  9884 raft_consensus.cc:993] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:50.694950  9822 raft_consensus.cc:1081] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:50.696322  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 12:59:50.696910  9448 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:50.697175  9448 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.701857  9448 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.703969  9448 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:50.704617  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:50.704963  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:50.705344  8823 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:50.705600  8898 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.705722  8823 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:50.706559  8823 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.707736  8823 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.710197  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:50.711282  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:50.712037  9879 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.712291  8823 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:50.713456  9879 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:50.714277  9879 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.723667  8647 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.919443  9884 raft_consensus.cc:993] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:50.919919  9884 raft_consensus.cc:1081] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:50.921459  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 12:59:50.921936  9448 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:50.922255  9448 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.926370  9448 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.928143  9448 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:50.928751  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:50.929358  8819 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:50.929684  8819 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:50.929754  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:50.930317  8897 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.930439  8819 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.931558  8819 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:50.935005  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:50.935860  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:50.936637  9879 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:50.937067  9879 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:50.937849  9879 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:50.936138  8819 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:50.946230  8647 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.130700  9900 raft_consensus.cc:993] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:51.131124  9829 raft_consensus.cc:1081] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:51.132457  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44b3244bf12f42aba79421575a96c621"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:59:51.132900  8823 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.133224  8823 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.134469  9884 raft_consensus.cc:993] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:51.134922  9822 raft_consensus.cc:1081] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:51.136325  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 12:59:51.136749  9448 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.137137  9448 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.139356  8823 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.141999  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:51.142174  9448 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.142515  8749 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.142609  8823 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:51.142892  8749 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:51.143635  9448 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:51.143658  8749 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.144502  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:51.144994  8897 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.145767  8749 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.145946  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:51.146499  8819 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.146790  8819 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:51.147420  8819 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.148221  8819 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.150911  8897 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:51.151717  8749 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:51.152020  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:51.152341  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:51.152750  9879 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.153203  9879 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:51.153158  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:51.154539  9822 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.154148  9879 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.156072  9822 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:51.156205  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:51.156749  8897 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.157217  9822 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.163887  8897 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:51.164584  9928 raft_consensus.cc:993] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:51.165479  8647 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.167598  9927 raft_consensus.cc:1081] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:51.174875  8897 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 12:59:51.175354  8897 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.175663  8897 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.176862  8643 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:51.181721  8897 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.186141  8897 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:51.186419  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:51.186342  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:51.186992  8819 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.187103  8749 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.187381  8819 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:51.188133  8819 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.189086  8819 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.192389  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:51.193593  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:51.194650  9893 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.195112  9893 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:51.195910  9893 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.200547  8819 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:51.206023  8643 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.229039  8819 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.230257  9879 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.242213  8897 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.244954  9937 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:51.428900  8823 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.430014  9937 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.440167  8898 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.441567  9879 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.559401  8740 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.559985  8897 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.560604  9932 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.561160  9879 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.582216  8823 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:51.584419  9924 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:51.604398  8897 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.605700  8749 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.607158  9893 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.609318  9932 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.635396  8819 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:51.641381  9820 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.646730  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:51.715364  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:51.731487  8749 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:51.739101  8713 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75401b4ca97a4c1eb75a016ff2124379. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:51.739679  8819 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.740940  9900 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.741683  8898 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.743953  9829 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.749104  9829 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.755023  8897 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.768646  8823 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.771896  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "029b57162d72422bba468c7d86b67fca"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:51.772462  8749 raft_consensus.cc:606] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:51.772953  8819 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:51.775485  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:51.776034  8820 raft_consensus.cc:606] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:51.776661  8647 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.788612  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:51.789183  8749 raft_consensus.cc:606] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:51.831405  8897 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.834314  9932 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:51.837613  8788 consensus_peers.cc:597] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 48fca36d2cac4f99b1a750bcc17b5d12. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:51.842779  8740 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.844312  9932 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.849067  9927 raft_consensus.cc:993] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:51.849566  9884 raft_consensus.cc:1081] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:51.851214  8897 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 12:59:51.851714  8897 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.852047  8897 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.859323  8897 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.861003  8897 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:51.862316  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:51.852182  9927 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.862879  8749 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.863409  8740 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.867736  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:51.868295  8819 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.868571  8819 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:51.869227  8819 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.870308  8819 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.878029  8749 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:51.879317  8897 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.880404  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:51.881985  9820 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.882409  9820 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:51.882632  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:51.883184  8898 raft_consensus.cc:606] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:51.883080  9820 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:51.887107  8819 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:51.889530  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:51.889003  8740 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:51.890748  8749 raft_consensus.cc:606] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:51.891804  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:51.893121  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:51.896435  8643 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.906993  9820 raft_consensus.cc:993] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:51.907990  9820 raft_consensus.cc:1081] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:51.909977  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:58306
I20260709 12:59:51.910466  9448 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.910769  9448 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.917572  9448 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.919800  9448 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:51.920614  8644 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:51.922317  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:51.924561  8749 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.927079  8898 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.927551  8819 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:51.928843  9953 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.929796  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:51.931185  8897 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:51.931473  8897 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:51.931850  9829 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:51.932226  8897 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.933338  8897 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:51.951737  8897 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:51.953984  8749 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
W20260709 12:59:51.954695  8713 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0eb392189d54431c863a3b787fb95779. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:51.954113  9412 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:51.952786  9953 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.955840  9924 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:51.956321  9924 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:51.957161  9924 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:51.963567  8819 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.970641  8897 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.983559  8644 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:51.989157  8749 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:51.999485  8646 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:52.012552  8713 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7bff3e7dddcd4dbebe38db1fa4eaae71. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:52.015174  8823 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.015379  8897 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.016938  9829 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:52.019201  9953 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:59:52.029419  9829 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:52.031647  8897 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:52.032580  8823 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:52.047335  8644 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.052925  9900 raft_consensus.cc:993] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:52.053483  9953 raft_consensus.cc:1081] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:52.057542  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:59:52.058060  8823 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.058367  8823 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.075166  9900 raft_consensus.cc:993] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:52.075683  9900 raft_consensus.cc:1081] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:52.075708  8823 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.077271  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "029b57162d72422bba468c7d86b67fca"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:58300
I20260709 12:59:52.077865  9448 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.078249  9448 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.084308  9448 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:52.086582  9448 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:52.086836  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:52.087057  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:52.087427  8749 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.087523  8819 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.087803  8749 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:52.088527  8749 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:52.089733  8749 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.090425  8823 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:52.097966  8819 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:52.098174  8749 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:52.098704  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:52.099434  8897 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.100661  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:52.101230  8742 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.101771  8742 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:52.102594  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:52.109573  8742 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.110924  8742 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.111008  9924 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.112210  9924 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:52.113046  9924 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:52.118997  8742 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:52.120672  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:52.121543  9927 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.122006  9927 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:52.122834  9927 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.129665  8897 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:52.134492  8643 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:52.135993  9934 raft_consensus.cc:993] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:52.136462  9934 raft_consensus.cc:1081] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:52.141521  8819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:59:52.142033  8819 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.142371  8819 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.145712  8644 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.148550  8819 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.150406  8819 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:52.151213  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:52.151733  8897 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.152750  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:52.153326  8749 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.153618  8749 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:52.154409  8749 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.156477  8749 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.157812  8897 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:52.159114  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:52.159950  9927 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.162773  8749 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:52.164903  9927 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:52.165861  9927 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:52.177235  8643 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.298265  9973 ts_tablet_manager.cc:933] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:52.300093  9973 tablet_copy_client.cc:323] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:52.301685  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 75401b4ca97a4c1eb75a016ff2124379 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:52.302229  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 75401b4ca97a4c1eb75a016ff2124379 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-75401b4ca97a4c1eb75a016ff2124379
I20260709 12:59:52.309762  8759 tablet_copy_source_session.cc:215] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.312757  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75401b4ca97a4c1eb75a016ff2124379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.326738  9973 tablet_copy_client.cc:806] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.327311  9973 tablet_copy_client.cc:670] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.329393  8819 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.330763  9820 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.331027  9973 tablet_copy_client.cc:538] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.341353  9973 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:52.369352  9977 ts_tablet_manager.cc:933] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:52.374246  8749 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.378583  9977 tablet_copy_client.cc:323] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:52.381384  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 0eb392189d54431c863a3b787fb95779 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:52.381989  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 0eb392189d54431c863a3b787fb95779 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-0eb392189d54431c863a3b787fb95779
I20260709 12:59:52.386801  9973 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.387779  9973 tablet_bootstrap.cc:492] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:52.388448  9973 ts_tablet_manager.cc:1403] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.000s
I20260709 12:59:52.391212  9973 raft_consensus.cc:359] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.392019  9973 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:52.393105  9973 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.394425  9820 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.398635  9973 ts_tablet_manager.cc:1434] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:59:52.400800  8897 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.401690  8758 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-75401b4ca97a4c1eb75a016ff2124379 received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:52.402117  8758 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-75401b4ca97a4c1eb75a016ff2124379 on tablet 75401b4ca97a4c1eb75a016ff2124379 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:52.402277  9924 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.423065  8759 tablet_copy_source_session.cc:215] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.433481  9977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb392189d54431c863a3b787fb95779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.438905  8742 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.440383  9924 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.445537  9973 ts_tablet_manager.cc:933] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:52.468250  9973 tablet_copy_client.cc:323] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:52.475893  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 48fca36d2cac4f99b1a750bcc17b5d12 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:52.476516  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 48fca36d2cac4f99b1a750bcc17b5d12 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-48fca36d2cac4f99b1a750bcc17b5d12
I20260709 12:59:52.499687  8834 tablet_copy_source_session.cc:215] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.517553  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fca36d2cac4f99b1a750bcc17b5d12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:52.521884  9983 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:59:52.545979  9981 ts_tablet_manager.cc:933] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 12:59:52.548904  9981 tablet_copy_client.cc:323] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
I20260709 12:59:52.552644  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 12:59:52.553148  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-7bff3e7dddcd4dbebe38db1fa4eaae71
I20260709 12:59:52.561205  8759 tablet_copy_source_session.cc:215] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.564445  9981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.571197  9977 tablet_copy_client.cc:806] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.571848  9977 tablet_copy_client.cc:670] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.577831  9973 tablet_copy_client.cc:806] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.578480  9973 tablet_copy_client.cc:670] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.582015  9977 tablet_copy_client.cc:538] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.587010  9973 tablet_copy_client.cc:538] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.592748  9977 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:52.601867  8819 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.603279  9879 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.603742  9973 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:52.612458  9981 tablet_copy_client.cc:806] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.613066  9981 tablet_copy_client.cc:670] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.617081  9981 tablet_copy_client.cc:538] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.628299  9981 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:52.637171  8749 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.641569  9973 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.642590  9973 tablet_bootstrap.cc:492] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:52.642863  9932 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.643261  9973 ts_tablet_manager.cc:1403] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.040s	user 0.026s	sys 0.007s
I20260709 12:59:52.646057  9977 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.646377  9973 raft_consensus.cc:359] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.649104  9973 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:52.649905  9973 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.658003  8740 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.660477  9879 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.660912  8897 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.661942  9977 tablet_bootstrap.cc:492] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:52.663123  9928 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.667245  9973 ts_tablet_manager.cc:1434] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 12:59:52.671356  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-48fca36d2cac4f99b1a750bcc17b5d12 received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:52.671782  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-48fca36d2cac4f99b1a750bcc17b5d12 on tablet 48fca36d2cac4f99b1a750bcc17b5d12 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:52.673774  9977 ts_tablet_manager.cc:1403] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.081s	user 0.017s	sys 0.016s
I20260709 12:59:52.686259  9977 raft_consensus.cc:359] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.690804  9977 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:52.691541  9977 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.714385  8740 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.715253  9977 ts_tablet_manager.cc:1434] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.002s
I20260709 12:59:52.717097  9932 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:52.717613  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-0eb392189d54431c863a3b787fb95779 received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:52.718132  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-0eb392189d54431c863a3b787fb95779 on tablet 0eb392189d54431c863a3b787fb95779 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:52.723618  9981 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.724664  9981 tablet_bootstrap.cc:492] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:52.746960  8897 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.748139  9927 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.755728  9448 raft_consensus.cc:1217] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:52.767534  9981 ts_tablet_manager.cc:1403] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.139s	user 0.025s	sys 0.009s
I20260709 12:59:52.771673  9981 raft_consensus.cc:359] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.772573  9981 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:52.773254  9981 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:52.776224  9981 ts_tablet_manager.cc:1434] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:59:52.777808  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-7bff3e7dddcd4dbebe38db1fa4eaae71 received from {username='slave'} at 127.0.0.1:45274
I20260709 12:59:52.778204  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-7bff3e7dddcd4dbebe38db1fa4eaae71 on tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:52.881209  9448 raft_consensus.cc:1217] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:52.893965  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:52.911463  9448 raft_consensus.cc:1217] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:52.918907  9448 raft_consensus.cc:1217] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:52.959105  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:53.009766  8742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a0644aef66444b58e8633da6993535c"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:53.010270  8742 raft_consensus.cc:606] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:53.012710  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:53.013319  8898 raft_consensus.cc:606] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:53.018553  8742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:53.019150  8742 raft_consensus.cc:606] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:53.023305  8823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:53.023795  8823 raft_consensus.cc:606] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:53.028487  8823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:53.029124  8823 raft_consensus.cc:606] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:53.032289  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:53.033991  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:53.076311  9927 raft_consensus.cc:993] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:53.076840  9927 raft_consensus.cc:1081] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:53.080204  8898 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 12:59:53.080757  8898 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.081090  8898 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.087306  8898 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.088979  8898 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:53.089920  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:53.089874  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:53.090595  8819 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.090673  8742 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.090960  8819 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:53.091727  8819 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.092813  8819 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.100054  8742 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:53.101138  8863 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:53.101255  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:53.101874  9820 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.102305  9820 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:53.103013  9820 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.114241  9932 raft_consensus.cc:993] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.113677  8646 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.114707  9927 raft_consensus.cc:1081] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.116163  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 12:59:53.116771  9448 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.117153  9448 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.123548  9448 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.132314  9448 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:53.133136  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:53.133914  8742 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.137576  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:53.139200  8819 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.140631  8819 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:53.141409  8819 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.142843  8742 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.142980  8819 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.144189  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:53.144963  9879 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.145484  9879 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:53.146327  9879 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.149945  8819 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.152652  9829 raft_consensus.cc:993] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:53.153052  9829 raft_consensus.cc:1081] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:53.155781  8643 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.156886  9900 raft_consensus.cc:1064] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:53.157541  8819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:59:53.158030  8819 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.159024  9900 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.162132  8819 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.164974  9448 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.166360  9829 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.168273  8819 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.171075  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:53.171562  8742 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.171593  8898 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.171943  8742 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:53.172621  8742 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.173569  8742 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.173435  9953 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.175418  8823 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.177196  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:53.177858  8898 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.184706  8742 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:53.186745  8789 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:53.187171  8898 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:53.187943  9932 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.188439  9932 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:53.189395  9900 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.189354  9932 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:53.177186  8819 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:53.252910  8897 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.249876 10005 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.261859  9900 raft_consensus.cc:993] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.262336  9900 raft_consensus.cc:1081] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.263832  9447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a0644aef66444b58e8633da6993535c"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:58300
I20260709 12:59:53.264375  9447 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.264710  9447 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.268436  9820 raft_consensus.cc:993] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.268952  9885 raft_consensus.cc:1081] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:53.269577  9446 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.275442  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:58306
I20260709 12:59:53.275944  9448 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.276273  9448 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.272965  9447 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.279132  9447 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:53.282994  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:53.283581  8895 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.280539  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:53.287561  8740 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.287850  8740 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:53.288498  8740 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.289534  8740 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.293708  8895 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.294853  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:53.300060 10010 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.300513 10010 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:53.301275 10010 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.310737  8643 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:53.313701  8740 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.315754  9448 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.320653  9448 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:53.322568  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:53.323084  8823 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.321534  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:53.327730  8646 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.336649  8895 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.332386  8823 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.335500  9900 raft_consensus.cc:1064] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:53.336972  8895 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:53.337702  8895 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.338781  8895 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.339133  9900 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.339349  8819 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.344058  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 12:59:53.344946  9937 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.356762  9932 raft_consensus.cc:1064] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:53.345250  8895 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:53.361496  9448 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.345407  9937 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:53.368202  9937 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.373680  9953 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.374805  8897 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.377548  8740 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.378283  9829 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.384882  8818 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.385232  9446 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.386317 10005 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:53.387212  9829 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:53.384908  9932 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.373929  8823 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.399991  9953 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.401543  9900 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.402477  9446 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.413040  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0eb392189d54431c863a3b787fb95779 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:53.430084  8647 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.441116  9900 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.447288  8819 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.456751  8647 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.458461  9446 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.458315  8647 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.474836  8894 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.483912  9953 raft_consensus.cc:1064] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:53.486276  9953 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.488121  8643 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.500202  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 0eb392189d54431c863a3b787fb95779 with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50476
W20260709 12:59:53.529026  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43609.6us
I20260709 12:59:53.534673  8823 raft_consensus.cc:2955] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.543556 10023 tablet_replica.cc:333] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:53.546485  8740 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.550446 10023 raft_consensus.cc:2243] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:53.553434 10023 raft_consensus.cc:2272] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:53.557904  8895 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.561980  8818 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.562793  9447 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.565172  9953 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.565377  9994 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.569746  8895 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.570879  9927 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.573969  9446 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.574501  9449 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.574440 10019 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:59:53.578507  8897 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.584470 10005 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:53.585816  8740 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:53.587083  9885 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:53.588326 10005 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:53.591864 10005 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.598698  9900 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.600732  8897 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.607803 10023 ts_tablet_manager.cc:1916] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:53.629583  8823 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:53.631520  9447 raft_consensus.cc:2955] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.635025  9893 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:53.633876  8643 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 12:59:53.701021  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 142585us
I20260709 12:59:53.712059 10019 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.714350  9928 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.720289  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:53.743877 10023 ts_tablet_manager.cc:1929] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:53.737950  9449 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.746201 10023 log.cc:1199] T 0eb392189d54431c863a3b787fb95779 P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/0eb392189d54431c863a3b787fb95779
I20260709 12:59:53.747298  9447 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.750413  8740 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.755774  8740 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.757035 10010 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.766307  8646 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.775815  8894 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.778450 10009 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.781052  8819 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.805887  8895 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.810082  8646 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.812565  9937 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.826292  8740 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.827597  8897 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:53.829478  8897 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.838363 10024 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:53.844556  8820 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.845988  8823 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:59:53.850539  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 204999us
I20260709 12:59:53.852027  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 0eb392189d54431c863a3b787fb95779 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:53.850953  8897 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.862430  8744 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:53.864216  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 85944.5us
I20260709 12:59:53.870074 10024 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.877038  8749 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.879019  9928 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:53.884200 10039 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:53.885684  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:53.888118 10047 tablet_replica.cc:333] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:53.891728 10009 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.892575 10046 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:53.897989  9447 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.893514  8824 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:53.907823 10022 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:59:53.962364 10047 raft_consensus.cc:2243] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:53.970973 10047 raft_consensus.cc:2272] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:53.972245  8896 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.977881 10022 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:53.979725  9449 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.001696 10047 ts_tablet_manager.cc:1916] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:54.018450  8633 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 75401b4ca97a4c1eb75a016ff2124379 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:54.032460  9447 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.035009  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.038358  8740 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.052830 10055 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:54.068287 10052 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.076226 10047 ts_tablet_manager.cc:1929] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.076817 10047 log.cc:1199] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/7bff3e7dddcd4dbebe38db1fa4eaae71
I20260709 12:59:54.077543  9448 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.081741  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 7bff3e7dddcd4dbebe38db1fa4eaae71 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:54.092157  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 48fca36d2cac4f99b1a750bcc17b5d12 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:54.102341  8646 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.108865  8896 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.113508  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet 75401b4ca97a4c1eb75a016ff2124379 with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:54.116313 10020 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:54.128168 10047 tablet_replica.cc:333] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:54.130242 10047 raft_consensus.cc:2243] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:54.130991 10047 raft_consensus.cc:2272] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:54.130748  8894 raft_consensus.cc:2955] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.138816  9932 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:54.140699 10047 ts_tablet_manager.cc:1916] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:54.145915  8646 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.154706  8895 raft_consensus.cc:2955] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:54.170271  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 48fca36d2cac4f99b1a750bcc17b5d12 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:54.196812 10047 ts_tablet_manager.cc:1929] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.197372 10047 log.cc:1199] T 75401b4ca97a4c1eb75a016ff2124379 P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/75401b4ca97a4c1eb75a016ff2124379
I20260709 12:59:54.198046 10060 tablet_replica.cc:333] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:54.198796  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet 75401b4ca97a4c1eb75a016ff2124379 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:54.199141 10060 raft_consensus.cc:2243] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:54.199935 10060 raft_consensus.cc:2272] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:54.201160  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.204375 10060 ts_tablet_manager.cc:1916] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:54.222677 10060 ts_tablet_manager.cc:1929] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.223260 10060 log.cc:1199] T 48fca36d2cac4f99b1a750bcc17b5d12 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/48fca36d2cac4f99b1a750bcc17b5d12
I20260709 12:59:54.225061  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 48fca36d2cac4f99b1a750bcc17b5d12 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:54.242384  8740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:54.242990  8740 raft_consensus.cc:606] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:54.245361  8895 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:54.245905  8895 raft_consensus.cc:606] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:54.248268  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:54.248730  8824 raft_consensus.cc:606] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:54.251055  8740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:54.251614  8740 raft_consensus.cc:606] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:54.254716  8740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:54.255146  8740 raft_consensus.cc:606] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 12:59:54.256141  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:54.257805  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:54.280350 10020 raft_consensus.cc:993] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:54.280920 10005 raft_consensus.cc:1081] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:54.282241  8895 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50492
I20260709 12:59:54.282661  8895 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.282922  8895 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.288463  8895 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.289865  8895 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:54.290804  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:54.292559  8824 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.291512  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:54.293883  8740 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.294242  8740 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:54.295940  8740 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.299993  8740 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.306737  8740 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:54.307898  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:54.308665 10042 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.309196 10042 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:54.309938 10042 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.313519  8824 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:54.321426  8644 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.450994 10034 raft_consensus.cc:993] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:54.451529 10042 raft_consensus.cc:1081] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:54.453016  8820 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 12:59:54.453488  8820 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.453759  8820 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.457906  8820 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.459609  8820 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:54.460618  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:54.461344  8747 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.460609  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:54.462006  8895 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.462327  8895 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:54.463027  8895 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.463999  8895 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.466820  8747 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:54.468425  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:54.469146  8895 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:54.469182 10057 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.469813 10057 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:54.470615 10057 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:54.480836  8644 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.532153  9927 raft_consensus.cc:993] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:54.532600 10057 raft_consensus.cc:1081] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:54.533881  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 12:59:54.534363  9448 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.534636  9448 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.539639  9448 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.541689  9448 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:54.541796  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:54.542505  8820 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.542577  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:54.542905  8820 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:54.543154  8895 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.543637  8820 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.544762  8820 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.545425 10020 raft_consensus.cc:993] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:54.545924 10005 raft_consensus.cc:1081] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 12:59:54.547322  9448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:58300
I20260709 12:59:54.547741  9448 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.548060  9448 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.549135  8895 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:54.550248  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:54.550851  8820 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:54.550987 10039 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.551491 10039 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:54.552253 10039 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.553686  9448 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.555509  9448 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:54.556376  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:54.556921  8747 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.556936  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:54.557420  8747 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:54.557667  8894 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.558183  8747 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.559131  8747 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.562527  8894 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:54.563520  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:54.564241 10009 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.564759 10009 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 12:59:54.565544 10009 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.566890  8747 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 12:59:54.568912  8644 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.581561  8644 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.617662 10020 raft_consensus.cc:993] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:54.618180 10005 raft_consensus.cc:1081] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:54.619642  8894 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50492
I20260709 12:59:54.620121  8894 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.621054  8894 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.627964  8894 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.630023  8894 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:54.631268  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:54.631821  8740 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.632124  8740 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:54.632344  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 12:59:54.632827  8740 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.634809  8740 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.632848  9448 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.640627  8740 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:54.642444  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:54.643183 10042 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.644160 10042 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:54.646377  9448 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:54.644948 10042 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:54.659080  8644 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.779752  8820 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.780937 10042 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:54.793239  8740 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.794657 10034 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:54.969357  8895 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:54.970566 10009 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:54.982170  8824 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:54.983377 10009 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.039100  8740 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.040325 10057 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.057058  8895 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.058720 10057 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.069550  8747 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.070770 10037 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.085721  9448 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.087221 10036 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.091886  8747 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.096590 10009 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.103786  8895 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.105302 10009 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.225790  8824 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 12:59:55.232239  8788 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4bb4c86bcc704ac2add360ac3e5823df. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.232625  8895 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.233400  8747 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.233830 10057 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.235723 10072 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.241753 10057 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.243374  8895 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.244103  8747 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.258606  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:55.265372  8824 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.264472  8643 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 12:59:55.280157  8788 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f2e6f6b24e84190b722a8e69bbe97fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.280781  8894 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.281179  8747 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.281989 10057 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.283252 10072 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.299470 10057 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.307281  8894 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.316115  8747 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.326259  8895 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.339290  8824 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.339861  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:59:55.342386  8862 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8087b240e9e4f5d9ea874980d0d7efa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.343307 10034 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.349120  8747 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.350530 10036 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.357501 10034 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.363255  8820 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.363677  8646 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 3, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.364176  8747 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.387874  8894 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.392050  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.410769  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "318340b5bf3b4640972d271299414f94"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:55.411335  8820 raft_consensus.cc:606] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:55.416733  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:55.417322  8747 raft_consensus.cc:606] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
W20260709 12:59:55.420521  8862 consensus_peers.cc:597] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 739742728baa463f9d9f5e5884dd2e30. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.422204  8820 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.423403  8896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:55.423954  8896 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:55.426489  8747 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.429119 10036 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:55.431037 10042 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.437357  8896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:55.437983  8896 raft_consensus.cc:606] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 12:59:55.439090  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:55.440845  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:55.449194 10042 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.451149  8824 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.452873  8747 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.459488 10037 raft_consensus.cc:993] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 12:59:55.459959 10036 raft_consensus.cc:1081] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 12:59:55.461768  8740 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:51680
I20260709 12:59:55.462246  8740 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.462566  8740 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.469000  8646 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 12:59:55.476123  8740 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.477861  8740 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:55.480540  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:55.481053  8896 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.481328  8896 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:55.482057  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 12:59:55.482576  9448 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.482031  8896 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.490272  8896 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.494396  9448 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 12:59:55.495450  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:55.496229 10005 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.496670  8896 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 12:59:55.496683 10005 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:55.497954 10005 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.515749  8646 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.638794  9927 raft_consensus.cc:993] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: : Instructing follower b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 12:59:55.639376 10057 raft_consensus.cc:1081] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 12:59:55.641108  8740 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "318340b5bf3b4640972d271299414f94"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:59:55.641716  8740 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.642089  8740 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.646586  8740 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:55.649896  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:55.650353  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:55.650965  8820 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.651242  8820 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:55.651919  8820 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:55.652884  8820 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.650409  8896 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.650488  8740 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:55.665345  8896 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 12:59:55.668774  8820 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 12:59:55.671941  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:55.672708 10005 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.675096 10005 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:55.675798 10005 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:55.688813  8643 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.700560 10098 ts_tablet_manager.cc:933] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:55.719514 10020 raft_consensus.cc:993] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:55.720070 10005 raft_consensus.cc:1081] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:55.720191 10098 tablet_copy_client.cc:323] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:55.722590  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 1f2e6f6b24e84190b722a8e69bbe97fa from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:55.722636  8820 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:59:55.723186  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 1f2e6f6b24e84190b722a8e69bbe97fa from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-1f2e6f6b24e84190b722a8e69bbe97fa
I20260709 12:59:55.723259  8820 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.723768  8820 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.729262  8820 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.731830  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 12:59:55.731886  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:55.732431  9448 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.732750  8740 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.733081  8740 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:55.733228  8834 tablet_copy_source_session.cc:215] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:55.733824  8740 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.734833  8740 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.738641  9448 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:55.739753  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 12:59:55.740530  9927 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.741003  9927 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:55.741539  8740 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:55.741788  9927 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:55.745494  8820 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:55.740518 10098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2e6f6b24e84190b722a8e69bbe97fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:55.752663  8643 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.763890 10098 tablet_copy_client.cc:806] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:55.764446 10098 tablet_copy_client.cc:670] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:55.771716 10098 tablet_copy_client.cc:538] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:55.778090 10098 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:55.809473 10098 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:55.810467 10098 tablet_bootstrap.cc:492] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:55.811251 10098 ts_tablet_manager.cc:1403] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.007s
I20260709 12:59:55.814091 10098 raft_consensus.cc:359] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.814881 10098 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:55.815471 10098 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.818204 10098 ts_tablet_manager.cc:1434] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.002s
I20260709 12:59:55.821831  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-1f2e6f6b24e84190b722a8e69bbe97fa received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:55.822237  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-1f2e6f6b24e84190b722a8e69bbe97fa on tablet 1f2e6f6b24e84190b722a8e69bbe97fa with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:55.854916 10098 ts_tablet_manager.cc:933] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:55.857206 10098 tablet_copy_client.cc:323] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 12:59:55.859002  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 4bb4c86bcc704ac2add360ac3e5823df from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 12:59:55.859525  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 4bb4c86bcc704ac2add360ac3e5823df from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-4bb4c86bcc704ac2add360ac3e5823df
I20260709 12:59:55.866803  8834 tablet_copy_source_session.cc:215] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:55.870522 10098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bb4c86bcc704ac2add360ac3e5823df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:55.886870 10103 ts_tablet_manager.cc:933] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:55.888645 10103 tablet_copy_client.cc:323] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:55.890252  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet d8087b240e9e4f5d9ea874980d0d7efa from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:55.890905  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet d8087b240e9e4f5d9ea874980d0d7efa from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-d8087b240e9e4f5d9ea874980d0d7efa
I20260709 12:59:55.894750 10098 tablet_copy_client.cc:806] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:55.895321 10098 tablet_copy_client.cc:670] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:55.897714  8908 tablet_copy_source_session.cc:215] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:55.899009 10098 tablet_copy_client.cc:538] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:55.900601 10103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8087b240e9e4f5d9ea874980d0d7efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:55.911419 10098 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:55.914247 10103 tablet_copy_client.cc:806] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:55.914849 10103 tablet_copy_client.cc:670] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:55.918870 10103 tablet_copy_client.cc:538] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:55.925048 10103 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:55.936040 10098 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:55.937268 10098 tablet_bootstrap.cc:492] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:55.939421 10098 ts_tablet_manager.cc:1403] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.005s
I20260709 12:59:55.942221 10098 raft_consensus.cc:359] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.943233 10098 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:55.943842 10098 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.947643 10098 ts_tablet_manager.cc:1434] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:59:55.949224  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-4bb4c86bcc704ac2add360ac3e5823df received from {username='slave'} at 127.0.0.1:36246
I20260709 12:59:55.949663  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-4bb4c86bcc704ac2add360ac3e5823df on tablet 4bb4c86bcc704ac2add360ac3e5823df with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:55.951764 10103 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:55.953166 10103 tablet_bootstrap.cc:492] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:55.953873 10103 ts_tablet_manager.cc:1403] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.004s
I20260709 12:59:55.956559 10103 raft_consensus.cc:359] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.957430 10103 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:55.958076 10103 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.963826 10103 ts_tablet_manager.cc:1434] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:59:55.965431  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-d8087b240e9e4f5d9ea874980d0d7efa received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:55.965834  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-d8087b240e9e4f5d9ea874980d0d7efa on tablet d8087b240e9e4f5d9ea874980d0d7efa with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:55.975950  9448 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.977150 10020 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:55.983419 10037 raft_consensus.cc:993] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:55.983819 10037 raft_consensus.cc:1081] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:55.985381  8820 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 12:59:55.985859  8820 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.986929 10103 ts_tablet_manager.cc:933] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:55.987730  8820 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.991725 10103 tablet_copy_client.cc:323] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 12:59:55.993220  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet 739742728baa463f9d9f5e5884dd2e30 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 12:59:55.993726  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet 739742728baa463f9d9f5e5884dd2e30 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-739742728baa463f9d9f5e5884dd2e30
I20260709 12:59:55.996003  8820 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:55.998021  8820 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:55.998607  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:55.999164  8747 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.999459  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:56.000008  8894 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.000264  8894 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:56.000999  8894 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.002437  8894 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.007592  8908 tablet_copy_source_session.cc:215] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:56.011085  8747 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:56.013482 10103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 739742728baa463f9d9f5e5884dd2e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:56.043162  8896 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.049304 10020 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.052037 10103 tablet_copy_client.cc:806] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:56.052985 10103 tablet_copy_client.cc:670] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:56.058549 10103 tablet_copy_client.cc:538] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:56.064679  8894 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:56.065891  8787 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:56.066680 10057 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.067148 10057 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:56.068012 10057 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.069450 10103 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 12:59:56.097149  8646 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.121697  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:56.122258  8824 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:59:56.122568 10112 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:59:56.132290  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:56.132843  8824 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:59:56.135520  8634 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for d8087b240e9e4f5d9ea874980d0d7efa already in progress [suppressed 2 similar messages]
I20260709 12:59:56.139235  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:56.139747  8824 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:59:56.142455 10103 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.143311 10103 tablet_bootstrap.cc:492] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 12:59:56.143895 10103 ts_tablet_manager.cc:1403] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.023s
W20260709 12:59:56.146307  8695 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:59:56.146754  8695 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for d8087b240e9e4f5d9ea874980d0d7efa already in progress
W20260709 12:59:56.147023  8695 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for d8087b240e9e4f5d9ea874980d0d7efa already in progress
I20260709 12:59:56.150499 10103 raft_consensus.cc:359] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.151151 10103 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 12:59:56.151641 10103 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.154387 10103 ts_tablet_manager.cc:1434] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:56.157418  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-739742728baa463f9d9f5e5884dd2e30 received from {username='slave'} at 127.0.0.1:40708
I20260709 12:59:56.157811  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-739742728baa463f9d9f5e5884dd2e30 on tablet 739742728baa463f9d9f5e5884dd2e30 with peer 112105225cc140459688be5d31dcb071
I20260709 12:59:56.171331  8896 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.172617 10111 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.179548  9448 raft_consensus.cc:1217] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:59:56.198757  8824 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.200205 10020 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.254382  8740 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.255733 10057 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.266896  9448 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.268757 10072 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.446336  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:56.540714  9448 raft_consensus.cc:1217] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:56.577950  9447 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Advancing to term 2
I20260709 12:59:56.582173  9449 raft_consensus.cc:1217] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 12:59:56.609025  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65316.4us
W20260709 12:59:56.613397  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72426.3us
I20260709 12:59:56.634192  9447 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:56.641533 10057 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:56.651984  8895 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:56.689181 10072 raft_consensus.cc:1064] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
W20260709 12:59:56.689742 10072 raft_consensus.cc:1068] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Unable to promote non-voter 112105225cc140459688be5d31dcb071: Service unavailable: leader transfer in progress
I20260709 12:59:56.695819  9927 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:56.709353 10072 raft_consensus.cc:1064] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:56.712196 10072 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:56.730918  8740 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:56.757686 10124 raft_consensus.cc:993] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:56.758175 10057 raft_consensus.cc:1081] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:56.761129  8895 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 12:59:56.761873  8895 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.762254  8895 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.771203  8895 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.773325  8895 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:56.774837 10122 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:56.775614  8896 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:56.778426  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:56.779103  8820 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:56.779419  8820 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:56.780371  8820 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.781827  8820 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.794764  8896 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.796134 10072 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.797747  9447 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.799834 10124 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.827735  8820 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
W20260709 12:59:56.830642  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 51760.8us
I20260709 12:59:56.837879  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:56.838549  8741 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.860253  8744 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.862751  8741 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 12:59:56.866202  8747 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:59:56.873917  8864 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:56.875551 10125 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:56.876056 10125 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:56.877025 10125 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 12:59:56.902031  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 3, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.906627 10072 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.914844 10122 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:56.918308  9449 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:56.929953 10072 raft_consensus.cc:1064] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:56.932756 10072 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:56.949575  8896 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:56.953332  8643 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.937479  8741 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:56.986883  8741 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.987231  9449 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.988727  9927 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.991191  8896 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.995862 10133 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.001361 10072 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.014936  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:57.040669 10133 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.049940  8896 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.063393  8747 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.070165  8820 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.084087  9447 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.084934  8747 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.086305 10122 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.087088  9448 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.088864 10122 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.093087 10129 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.096807  8643 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 3 to 4, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.098349  9446 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.104643  8744 raft_consensus.cc:2955] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.126849  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f2e6f6b24e84190b722a8e69bbe97fa with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.134853 10125 raft_consensus.cc:1064] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:57.136978  8646 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 4 to 5, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.137920 10125 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.152982  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f2e6f6b24e84190b722a8e69bbe97fa with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50476
I20260709 12:59:57.155465  8820 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.162645  9448 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.164316 10133 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:57.168980 10146 tablet_replica.cc:333] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 12:59:57.171818 10072 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.172286 10146 raft_consensus.cc:2243] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.173694 10146 raft_consensus.cc:2272] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.177762 10146 ts_tablet_manager.cc:1916] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.180292  9448 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.184294  8818 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.185774  9446 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.187342  8744 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.190377 10125 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.191449 10132 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.195592 10036 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.197024  8896 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.198396  9927 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.211074  8896 raft_consensus.cc:2955] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.246927  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4bb4c86bcc704ac2add360ac3e5823df with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.252931 10036 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.259743  8646 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 4 to 5, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:57.262996  8744 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.270313 10146 ts_tablet_manager.cc:1929] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:57.270951 10146 log.cc:1199] T 1f2e6f6b24e84190b722a8e69bbe97fa P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/1f2e6f6b24e84190b722a8e69bbe97fa
I20260709 12:59:57.275336  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet 1f2e6f6b24e84190b722a8e69bbe97fa (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:57.292636  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0192ad465a724ab581b9529345f73bba"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 12:59:57.293260  8824 raft_consensus.cc:606] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:57.298071  8895 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 12:59:57.298727  8895 raft_consensus.cc:606] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 12:59:57.302443  8744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:32798
I20260709 12:59:57.303146  8744 raft_consensus.cc:606] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 12:59:57.305202  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:59:57.306770  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:57.321785  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 4bb4c86bcc704ac2add360ac3e5823df with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:57.327260 10152 tablet_replica.cc:333] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:57.326494  8644 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.328684 10152 raft_consensus.cc:2243] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.328455  9448 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.329603 10152 raft_consensus.cc:2272] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.347071 10152 ts_tablet_manager.cc:1916] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.345314  8824 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.368893 10122 raft_consensus.cc:993] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:57.369375 10122 raft_consensus.cc:1081] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:57.372206  8895 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0192ad465a724ab581b9529345f73bba"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 12:59:57.372725  8895 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.373066  8895 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.374959 10152 ts_tablet_manager.cc:1929] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:57.375469 10152 log.cc:1199] T 4bb4c86bcc704ac2add360ac3e5823df P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/4bb4c86bcc704ac2add360ac3e5823df
I20260709 12:59:57.377525  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 4bb4c86bcc704ac2add360ac3e5823df (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:57.378672  8741 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:57.381320  8896 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.384323  8895 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.386078  8895 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:57.387315  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 12:59:57.381259 10125 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:59:57.388252  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 12:59:57.388785  9448 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.387844  8824 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.389222  8824 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:57.390758  8824 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.391964  8824 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.392191  8820 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.392191  9446 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.396108 10125 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.397505 10132 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.398676  9446 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 2 LEARNER]: Advancing to term 3
I20260709 12:59:57.415445 10125 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.416600  8820 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.428360  9446 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:57.431877 10036 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.438292  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 739742728baa463f9d9f5e5884dd2e30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.402295  9448 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:57.446645  8644 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 1 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.449116  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:57.449939 10125 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.450587 10125 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:57.451471 10125 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.451504  9448 raft_consensus.cc:2955] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.461998  8731 tablet_service.cc:1558] Processing DeleteTablet for tablet 739742728baa463f9d9f5e5884dd2e30 with delete_type TABLET_DATA_TOMBSTONED (TS b60591217ee24378bb66a86d6d0dda52 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51658
I20260709 12:59:57.464131 10152 tablet_replica.cc:333] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 12:59:57.465430 10152 raft_consensus.cc:2243] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.469101  8823 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:57.470783 10036 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.456681  8824 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:57.493700 10152 raft_consensus.cc:2272] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.526487 10152 ts_tablet_manager.cc:1916] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.562245  8646 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.574527 10036 raft_consensus.cc:1064] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 12:59:57.577091 10036 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.585397  8824 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:57.587342 10159 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.589461  8744 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:57.590763 10125 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.591948 10160 raft_consensus.cc:993] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:57.594576  9448 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:59:57.595197 10159 raft_consensus.cc:1081] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 12:59:57.597476  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 12:59:57.598059  8823 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.600147  8823 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.606580 10036 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.611524  8747 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.615538 10125 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.618221 10152 ts_tablet_manager.cc:1929] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:57.618681 10152 log.cc:1199] T 739742728baa463f9d9f5e5884dd2e30 P b60591217ee24378bb66a86d6d0dda52: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/wals/739742728baa463f9d9f5e5884dd2e30
I20260709 12:59:57.620332  8633 catalog_manager.cc:5028] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): tablet 739742728baa463f9d9f5e5884dd2e30 (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:57.622351  9448 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.620070  8823 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:57.629835  8823 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:57.633760  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 12:59:57.635030  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:57.639034  8894 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.634195  8823 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.639050  8744 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.640244  8894 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:57.641348  8894 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:57.642669  8894 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.634846  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 2 to 5, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.646245  8744 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:57.647212  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 12:59:57.647972  9927 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.648370  9927 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 12:59:57.649155  9927 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:57.653075  8894 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 12:59:57.672319  8894 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.676947  8747 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:57.677939  9448 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:57.678062 10159 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:59:57.679811  8646 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.682518 10159 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:59:57.684473 10128 raft_consensus.cc:993] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:57.685081 10150 raft_consensus.cc:1081] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 12:59:57.686736  8894 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50492
I20260709 12:59:57.687286  8894 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.687691  8894 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.686962 10132 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.693945  8894 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.692612  9448 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.696954  8894 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:57.699391  8631 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8087b240e9e4f5d9ea874980d0d7efa with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.702351  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 12:59:57.702889  9446 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.705694  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 12:59:57.706295  8747 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.706585  8747 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 12:59:57.707283  8747 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.708305  8747 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.708803  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 5 to 6, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.710302  9446 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:57.711722  8744 raft_consensus.cc:2955] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 12:59:57.716437  8747 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 12:59:57.717432  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 12:59:57.718362 10132 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.719065 10132 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 12:59:57.719873 10132 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:57.739199  8809 tablet_service.cc:1558] Processing DeleteTablet for tablet d8087b240e9e4f5d9ea874980d0d7efa with delete_type TABLET_DATA_TOMBSTONED (TS 29c7934ee528454785432732d01ed0a0 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37548
I20260709 12:59:57.741098 10163 tablet_replica.cc:333] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 12:59:57.750285 10163 raft_consensus.cc:2243] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.753382 10163 raft_consensus.cc:2272] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.754106  8647 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.756258 10163 ts_tablet_manager.cc:1916] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.770632 10163 ts_tablet_manager.cc:1929] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:59:57.771088 10163 log.cc:1199] T d8087b240e9e4f5d9ea874980d0d7efa P 29c7934ee528454785432732d01ed0a0: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/wals/d8087b240e9e4f5d9ea874980d0d7efa
I20260709 12:59:57.772691  8634 catalog_manager.cc:5028] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): tablet d8087b240e9e4f5d9ea874980d0d7efa (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 12:59:57.883134  8824 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.884290 10132 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.893239  9446 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.895212 10125 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.117591  8744 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:58.119066 10122 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:58.145537  8894 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:58.146729  9927 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:58.179397  8747 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.180428 10132 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.187963  9446 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.189256 10036 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.307677  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:58.485913  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.356178 10171 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 12:59:59.356993 10171 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.359719 10171 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:59.508368 10171 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.509182 10171 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.512444 10171 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.612367 10171 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.613209 10171 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.622419 10171 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 12:59:59.711030  7643 debug-util.cc:398] Leaking SignalData structure 0x7b080000f300 after lost signal to thread 8789
W20260709 12:59:59.797390  7643 kernel_stack_watchdog.cc:198] Thread 8789 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 279ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:59:59.799165 10171 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.801280 10171 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.812062 10178 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 12:59:59.815582 10178 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.818280 10178 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.807479 10171 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 12:59:59.824990  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.393s	user 0.011s	sys 0.011s
W20260709 12:59:59.826133  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.395s	user 0.012s	sys 0.011s
I20260709 12:59:59.830629 10177 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.831138 10177 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.831640 10174 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 12:59:59.832424 10174 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.833204 10177 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.835785 10175 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.836438 10175 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.822647 10171 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 12:59:59.840221 10179 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.842602 10174 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:59.861716 10182 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.863983 10182 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.851877 10177 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.864174 10174 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.869148 10174 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.871094 10178 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.843335 10171 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.853235 10179 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.887183 10185 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.887725 10185 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.888410 10179 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.871847 10178 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.889380 10180 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.889915 10180 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.891147 10178 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.875850 10175 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 12:59:59.891873 10180 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.873383 10174 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.879981 10187 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.886170 10182 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.875294 10177 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.895177 10187 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.898651 10187 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.899163 10171 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.897065 10177 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.901700 10185 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.906832 10188 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 12:59:59.907714 10188 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.910360  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 93) took 1518 ms. Trace:
I20260709 12:59:59.911427 10189 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.910858  8744 rpcz_store.cc:276] 0709 12:59:58.391775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:58.391913 (+   138us) service_pool.cc:224] Handling call
0709 12:59:58.392203 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:59:58.393715 (+  1512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:59:58.393741 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:59:58.393779 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:59:58.393808 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:59:58.393861 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:59:59.910223 (+1516362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:59.915323 10185 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.917872 10177 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 12:59:59.918048 10181 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.918491 10177 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.918633 10181 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.920948 10177 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.923673 10182 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.924155 10182 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.924569 10180 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.925076 10180 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.925832  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.926126 10182 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.926640 10182 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.926985 10180 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.927058 10182 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.927534 10180 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 12:59:59.928551 10177 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.929050 10177 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.929096 10182 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.931093 10177 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.928072 10180 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.933274 10180 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.933895  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.934501  9448 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:59.936259  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.940212  9448 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:59.942945 10181 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.945801  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.938228 10184 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 12:59:59.947152 10184 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.951632  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:59.943150 10178 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.952591 10178 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.953624  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:59.954478  8824 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 12:59:59.955133 10178 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.955847  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:59.957500  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 12:59:59.958014  8824 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 12:59:59.960999 10184 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.962008  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.962824  9444 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:59.964936 10187 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.965471 10171 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.965603 10187 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.966017 10171 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.976188 10178 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.976826 10178 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.977308 10174 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.915855 10185 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.977818 10174 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.978972 10178 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.979250 10185 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.979395 10175 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.979990 10175 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.980185  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 12:59:59.981031  9443 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 12:59:59.981868 10175 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.982313 10175 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.982717 10175 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 12:59:59.984475 10175 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 12:59:59.984870 10175 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 12:59:59.911916 10188 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 12:59:59.979985 10177 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 12:59:59.985250 10175 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.032541 10177 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.032902 10009 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.021411 10180 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.033535 10009 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.033942 10175 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.033805 10180 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.994796 10185 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.034627 10177 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.034853 10185 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.035874 10180 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.036016 10009 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.021646 10182 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.036813 10185 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.037000 10182 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 12:59:59.991205 10178 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.038050 10178 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.039155 10182 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.039602 10181 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.040361  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.040184 10181 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.040503 10184 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.044909 10184 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.047086 10184 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.030880 10187 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.037482 10174 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.048774 10196 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.029881  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.043468 10176 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.049845 10176 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.051882 10176 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.052089 10184 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.052820  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.043887  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.048998 10181 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.042100 10188 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.054324 10188 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.054723  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 12:59:59.911957 10189 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.055193 10194 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.031170 10171 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.056700 10182 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.052700 10184 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.058238 10196 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.060097 10192 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.054652 10185 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.065232 10201 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.040207 10178 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.042860  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.076234  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.066515 10193 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.091429 10202 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.080050 10197 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.056713 10188 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.090423  8820 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.082330 10185 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.090435  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.070901 10194 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:00.100093  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45429.6us
I20260709 13:00:00.101017 10200 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.105026 10183 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.105692 10186 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.106223 10190 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.107622  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.091653 10193 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.104008 10185 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.106962 10178 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.080704 10182 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.086745 10192 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.107095 10194 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.116820 10208 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.091970 10202 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.093261 10201 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.111035 10197 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.112917 10188 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.115855 10200 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.117101 10186 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.290588 10217 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:00.278774  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800090820 after lost signal to thread 7641
I20260709 13:00:00.296593 10213 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.297405 10205 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.298138 10184 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.298333  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.266733 10199 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.298714 10200 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.115689 10180 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.117719 10178 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.302667 10214 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.183450 10192 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.182574 10189 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.118583  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.179148  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.271436 10195 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.278211 10191 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.307581  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.279042 10211 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.181766 10204 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.307211 10196 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.310722 10180 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.317262 10219 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.319980  9444 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:00.322122  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.309100 10184 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.339649  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.340842  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.344458  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.299736 10188 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.312342 10174 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.347457  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.183686  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.315688 10221 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.347466 10171 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.157932 10190 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.320328 10199 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.315019 10178 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.312222 10200 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.350710 10180 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.350898 10216 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.158111 10183 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:00.349695  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800090f00 after lost signal to thread 8681
I20260709 13:00:00.354911  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.355543  9447 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:00.350980 10223 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.360073 10171 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.348462 10212 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.371304  8741 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:00.304152 10208 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.348496  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.320597 10191 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.355921 10187 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.365238  8896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0192ad465a724ab581b9529345f73bba"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:00:00.376579  8896 raft_consensus.cc:606] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:00:00.361743  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.358096 10174 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.358016  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.365136 10210 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.360515 10193 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.312463 10217 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.383541  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.372005 10171 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.368429 10182 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.347510 10214 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.376389 10187 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.379797 10174 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.355710 10219 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.356418 10184 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.311812 10205 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.372232 10009 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.387847  8740 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.320685 10195 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.377637 10196 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.381956 10189 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.383749 10222 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.379958 10186 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.370203 10200 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.360869 10221 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.395278 10187 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.352003 10213 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.394937  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.384153 10223 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.387473 10192 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.420010  8747 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:00.386099 10193 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.423781  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.356102 10211 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.455577  8742 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.435223 10225 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.353034 10204 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.419405 10203 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.388262 10215 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.419553 10009 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.423671 10189 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:00.434882  8678 debug-util.cc:398] Leaking SignalData structure 0x7b080033f820 after lost signal to thread 8773
I20260709 13:00:00.395586 10210 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.463724 10218 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.466055  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.456105  9445 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:00.463665 10215 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.377395 10216 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.438452 10196 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.462767 10189 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.462621 10009 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.482215  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.482455 10198 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.456571 10192 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.441670 10222 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.381662 10212 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.468606 10188 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.494592 10215 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.496771 10210 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.465358 10202 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.469352 10197 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.480481 10201 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.501791 10220 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:00.465977 10218 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.506677 10216 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:00.460469 10193 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.466007  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:00.517458  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 161317us
W20260709 13:00:00.503913  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08002a18a0 after lost signal to thread 8848
I20260709 13:00:00.515578 10215 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.466125 10203 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.523248 10200 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.518537 10199 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.534713  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.516409 10217 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.536674  8742 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:00.482987 10198 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:00.538617  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 73254.6us
I20260709 13:00:00.530753 10184 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.516582 10227 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.532004  8745 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.532569 10181 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.551378  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:00.466472 10225 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.539969 10200 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.538004 10190 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.564330 10205 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.566110  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:00.567519 10171 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.568146  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:00.552207  8678 debug-util.cc:398] Leaking SignalData structure 0x7b080029e020 after lost signal to thread 8922
I20260709 13:00:00.541061 10202 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.550055 10204 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.550710 10199 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.532418 10215 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.549980  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.579633  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.580915  8823 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:00.552794 10211 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.563200 10184 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.569474 10217 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.595467  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.598419  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.599869 10209 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.603957  9449 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.556751 10181 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.616648  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.599260 10233 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.621213  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.598424 10232 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.618640  8746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:32798
I20260709 13:00:00.592167  8748 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:00.549659 10226 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.580304 10200 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.594476 10184 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.624583 10236 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.592350 10216 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.589483 10202 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.626600  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.613834 10198 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.536976 10220 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.594337 10199 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.621366 10213 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.650009 10216 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.627732  8743 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:00.646054 10196 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.645411 10230 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.627370 10222 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.654120  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.651664 10221 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.645896  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.644917  8746 raft_consensus.cc:606] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:00.653156  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.627213 10201 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.611353 10217 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.662317  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.692582  8895 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:00.662920 10212 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.646064 10184 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.646508 10192 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.625273 10191 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:00.652032  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 127586us
I20260709 13:00:00.653000 10226 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.649092 10223 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.672832 10225 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.653824 10236 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.677624  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.563127 10227 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.679355 10206 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:00.679406  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.682472 10178 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.684703  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.697789  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.690487 10200 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.674347  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.697057 10220 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.648362 10209 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.697778  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.706930  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:00.702491 10185 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.000s
W20260709 13:00:00.687538  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:00.753823  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:00.708786  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800345f00 after lost signal to thread 9473
I20260709 13:00:00.738063  8749 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:00.646684 10232 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:00.708391  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 142444us
I20260709 13:00:00.695188 10009 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.696834  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.734377 10236 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.791462  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.792078  8740 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:00.750699  8895 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:00.749979 10181 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.702015 10201 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.753525 10185 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.792578  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:00.647709 10233 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.793656  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.796627  8896 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:00.788635 10174 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.705191 10230 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:00.798211 10234 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:00:00.802578 10009 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.748139 10171 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.791141 10178 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.805455  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.745249  8893 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:00.809896  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.810714  9447 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:00.808557  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:00.813002  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.793933 10203 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.815173  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.806713 10215 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.817660  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.809412  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:00.824882  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.798508 10206 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:00.825080  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.879246 10209 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.830437  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.827860 10216 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.835634  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.889616  9446 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:00.879021  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.833608  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.823987 10199 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.839658  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:00.827574  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.891989  8892 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:00.879498 10232 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.898754 10236 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.891271 10201 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.879941  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:00.894407  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:00.903051 10194 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.004s
I20260709 13:00:00.890607 10233 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.840004  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:00.912021  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76297.2us
I20260709 13:00:00.912122 10171 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:00.912640 10171 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.912729  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:00.882462 10193 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.871850 10226 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.916558  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.828250  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.913270 10171 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.915234 10235 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.924791 10235 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.925364 10235 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.915912  8815 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.916038 10187 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.936002 10187 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:00.868960 10238 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.872555 10185 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:00.937100 10238 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:00.937664 10238 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.939756  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:00.838474 10217 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:00.826640 10180 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:00.951754  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:00.952186  8824 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.952394  8818 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.960908  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:00.962497  9448 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.852521 10230 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.005635  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:01.006099  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.009581  8895 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:01.011559 10216 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.011952 10216 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:00.903462 10194 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.016005  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.019043  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.019341 10180 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.019903  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.020186  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.020493  8822 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:01.021656  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.028937 10194 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.030030  8816 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.904363 10189 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.012266 10216 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.913960 10184 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.071597  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.091171  8815 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:01.093120 10195 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.093910 10202 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.070001  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.169404  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:01.125351  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 107882us
I20260709 13:00:01.169207  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.173802  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.174292  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.175309  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.177875  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.068802 10174 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.178807  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.181780  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.182143  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.182761  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.094368  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.125463  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.184257  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.186605  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.187248  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.108462 10219 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.108479  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.124332  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.095726 10208 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.116397 10206 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.188306  8822 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:01.195676 10214 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.195749  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.196019 10218 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.196367  8820 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:01.196717  9442 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.186758 10199 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.189342  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.118848 10171 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.192443  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.193101  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.179898 10236 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.178360 10229 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:00:01.206301  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.203313  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:01.205525  8713 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4f573b86fda4b3e8fd982da9615dc8e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:01.194633  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.199384 10199 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:01.212982  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.215582  8820 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.216794  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.218212  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:01.220943 10183 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.222064  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.224452  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:01.220240  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.226287  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.226934 10223 raft_consensus.cc:993] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 13:00:01.227502 10232 raft_consensus.cc:1081] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 13:00:01.227571  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.228358  9443 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 2: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:01.228449 10238 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.229027  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.229099 10184 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.230752 10171 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.231669  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.232494 10183 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.232808  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.232899 10183 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:01.233460 10183 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.233942  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.235013  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.234790 10235 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.236830 10235 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.214535 10199 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.214754  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.245966  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.246665  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.244925 10206 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.248983  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.251799 10193 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.252784  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.220118 10189 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.254424  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.254722  9444 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.255818 10193 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.257145  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.257680 10193 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.258368  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.255566  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.262020 10208 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.262382 10208 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.262755 10208 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.262800  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.263828  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.267314  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.248298 10182 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.276361  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.253585 10232 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.301589  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.253685 10206 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.317485 10235 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.321338  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.262020  9444 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:01.321396  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65309.4us
I20260709 13:00:01.326808  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.326642  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.328182  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.256160  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:01.259274  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.329723  9447 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.321231  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.318403 10189 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.318749 10232 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.337774  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.277305 10182 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:01.350390  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96006.7us
I20260709 13:00:01.339380  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.347102  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.321762 10206 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.352021  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.317375 10233 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:00:01.354473 10233 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.354818 10233 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.355754 10233 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.357340  8820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:01.352651  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.357962  8820 raft_consensus.cc:606] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 13:00:01.361630  8748 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:01.361344  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.356997 10233 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.364300  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.391722  8895 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:01.329774 10203 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:00:01.362529  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.393028  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.366830  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.353204 10182 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.395586  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.358639 10197 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.369222  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.372504 10229 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.372830  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.416177  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.416548  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.416734  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.373903  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:01.374416  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45933.5us
I20260709 13:00:01.374433 10221 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:01.375118 10202 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.375365 10219 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.376796 10201 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.379262 10217 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.430550  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.380694  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:01.431581  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57752.6us
I20260709 13:00:01.382053 10227 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.382361  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.383937  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.384939  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.388717  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.351362 10232 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.367394  8818 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:01.367844  9445 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:01.397255  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.398185  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.350189  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.393810  8898 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:01.409598  8821 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:01.373488  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:00:01.362704 10223 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.455417 10223 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.455782 10223 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.423017  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.427718  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.456763  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:01.453598  8749 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
W20260709 13:00:01.430655  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 51530.2us
I20260709 13:00:01.379796  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.432286  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.424652 10219 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:01.436429  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.437701  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.438786  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.409741  8745 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:01.447793  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.450143  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.478258 10238 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.452759  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.481385  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.482630  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.483057  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.458662  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.439729 10230 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.484753 10230 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.485342  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.486056 10189 raft_consensus.cc:993] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 112105225cc140459688be5d31dcb071 to start an election
I20260709 13:00:01.461218  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.486631 10189 raft_consensus.cc:1081] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Signalling peer 112105225cc140459688be5d31dcb071 to start an election
I20260709 13:00:01.487560  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.487888 10229 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.488148 10231 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.488225 10229 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:01.488677  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.489280 10231 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.489553 10197 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.489776 10224 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.490164 10224 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.490944 10224 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.492237 10224 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.463125  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.492187 10216 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.493373 10244 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:01.494308 10244 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:01.494626  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114890us
I20260709 13:00:01.495384  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.460464  9441 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.496541  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.497154  8895 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:01.497848 10174 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.497848  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.498375 10219 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.499632 10219 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.466446  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.467206  8746 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:01.418082 10199 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.469187 10218 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.504572 10199 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.504725  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.505518 10231 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.505558  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.506007 10231 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:01.505883 10183 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.506476 10231 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.508042 10183 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.508683  8896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50492
I20260709 13:00:01.509469  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.509852  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.510468  8746 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:01.511166  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.511411  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.511787  8745 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:01.512523  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.512655  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.513646  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.513899  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.514856  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.516052  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.517097  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.456115  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.470175  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:01.519019  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48928.7us
I20260709 13:00:01.519371 10235 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.519905 10235 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:01.520195  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:01.498404 10174 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:01.471439  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.521152 10174 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.520768 10235 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.443838  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.521154 10181 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.522318 10181 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.473793 10217 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.522687 10181 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.522888 10217 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.474472 10175 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.527105 10175 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.527519 10175 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.485630 10230 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.452126 10184 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.461225  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.489315 10229 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.521559 10218 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.524652  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.497169 10197 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.538335  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.532274 10243 ts_tablet_manager.cc:933] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:01.524667  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:01.494626 10244 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.552731  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.546140  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.499282  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.549959  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.550195  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.550762  8896 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.544365  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.561887  8896 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.532789 10218 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.542762 10199 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.489092 10193 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.544622 10214 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.540664 10197 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:01.546136 10230 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.563621  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.564225  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.566808  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.548753  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:01.569535  8789 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.518s (SENT)
I20260709 13:00:01.572844  8740 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.548202 10184 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:00:01.531769 10217 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.566268 10218 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.574636  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.569525  9441 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.574101  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.571566 10197 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.572664 10171 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.583876  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.580355  8740 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.581655  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.575991 10199 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.586325  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:01.575455  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76245.1us
I20260709 13:00:01.585493 10238 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.595176  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.596798  8823 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.584089  8748 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.593091 10244 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.584266 10171 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:01.619791 10244 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:01.619848 10171 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.622735 10201 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.602886 10217 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.639122  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.647631 10239 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.648952 10245 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.649484 10245 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.657509  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.659942  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.668512  8896 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.707551  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.708505  8820 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.650513 10245 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.711213  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.713418  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.669538 10216 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.670074 10193 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.715564  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.715804  8896 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.717113  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:01.720763  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:01.726786  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.732395  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.733034  9443 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.737917  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.738456  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.742858  8896 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:01.748566  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:01.748342  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 178747us
I20260709 13:00:01.754012  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.756655  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.747126 10226 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.747859  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.748987 10245 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.751338 10235 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.767988  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.757918 10174 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.775720  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.757961 10231 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:01.757916  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47752us
I20260709 13:00:01.759052 10226 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.773177 10174 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.781634 10231 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:01.782173 10208 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.005s
I20260709 13:00:01.752381 10243 tablet_copy_client.cc:323] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.129:39991
W20260709 13:00:01.789136  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.752664 10206 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.766294 10181 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.782346  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.805475  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.806568  8743 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.777763  8748 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:01.809392 10181 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.810259  8740 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.810209  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.787801  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.815366  9443 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:01.801957  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.802228 10208 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.817185 10175 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.819018 10208 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.819352  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.819901  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.780438  9441 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.803947  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.819516 10175 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.785758  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.819051  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.828773  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.823129  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.830905  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.831576 10194 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.826828 10182 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.832456  8819 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.833658 10182 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:01.834383 10183 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.835981 10171 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.834532  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.837458 10226 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.838253 10171 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.838558 10194 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.838873 10226 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.842913  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.803293 10206 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.848459  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.852066  9443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8"
dest_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:44010
I20260709 13:00:01.852685  9443 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.853087  9443 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:01.853591  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:01.854280  9448 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:01.825657  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.859521  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:01.860132  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56308.4us
I20260709 13:00:01.862581 10231 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.862210  8646 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.868597 10229 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.868994  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.872857  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.878564  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.881615  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.883386 10174 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:01.885100  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.885439  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.885545 10218 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.886111  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.886323 10174 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.886888 10174 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:01.887600 10218 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.887727 10174 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.889518 10216 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.894217 10232 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.894923  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:01.895247  8747 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.896080  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.898113  9448 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.898411 10201 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:01.898923 10201 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.903586 10233 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.896272 10232 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.900146  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.905683 10233 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.889154  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.906665  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.907043  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.906032 10233 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:01.908102  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.908759 10233 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.910074 10209 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.910504  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.913466 10209 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.911552 10201 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.918191  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:01.918419  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.915371  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:01.919822  9448 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.913453  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.923906  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.925981  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.872306 10223 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:01.929150  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:01.927806  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:01.933211 10245 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.929656  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.935052  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.930714 10233 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.931326  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:01.936499  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.926826 10219 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:00:01.939453 10219 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.939780 10219 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:01.940549 10219 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.941565  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.941717  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.942873  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.943962  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:01.945057  8744 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.932024 10194 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:00:01.945947  8742 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:01.946192 10194 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.946590 10194 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:01.947467 10219 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.952260  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.953183  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:01.947381 10194 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.955317 10194 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.956230  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.956382 10180 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.956776 10180 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:01.948982 10223 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:01.957141 10180 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.957440 10245 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.949667  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.926888 10209 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.958042  9443 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.959125  9447 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:01.949905  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.950595  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:01.953243  8898 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:01.949242  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:01.962605 10194 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.963953  8746 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:01.965438  9443 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:01.893177 10197 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:01.970968 10171 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.980217  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.982396  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.978804  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:01.981094  8819 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:01.998310  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.000077  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.001174  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.003243  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.005370  8759 tablet_copy_service.cc:140] P b60591217ee24378bb66a86d6d0dda52: Received BeginTabletCopySession request for tablet d4f573b86fda4b3e8fd982da9615dc8e from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:45274)
I20260709 13:00:02.006106  8759 tablet_copy_service.cc:161] P b60591217ee24378bb66a86d6d0dda52: Beginning new tablet copy session on tablet d4f573b86fda4b3e8fd982da9615dc8e from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:45274: session id = 112105225cc140459688be5d31dcb071-d4f573b86fda4b3e8fd982da9615dc8e
I20260709 13:00:02.007532  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.009428 10239 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.010056 10239 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.011067  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.011324  9443 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.011896  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.012748 10223 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.990228 10233 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:00:02.011637  8749 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:02.015107 10209 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:01.997304  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:02.024471  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.024801  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.025346  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.027309  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.022244  9441 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.017133  8748 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:02.034992  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.037751  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.039665  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.040942  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.045425  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.046615  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.047405  8741 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:02.030756 10206 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.045499  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.034441 10224 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:00:02.055564 10180 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.058117 10226 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.060497  8749 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:02.061849  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.062088  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:02.058175 10201 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.065299 10201 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.066097 10201 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.068104 10226 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.068912 10226 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:02.071120  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57122.4us
W20260709 13:00:02.080866  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45902.3us
W20260709 13:00:02.085150  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 83990.1us
I20260709 13:00:02.087172  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.088304 10191 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.049602 10223 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.092885  8759 tablet_copy_source_session.cc:215] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:02.093506 10193 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:02.069818 10238 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.098485 10197 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.079816 10231 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:02.101025 10231 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:02.102067 10231 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.124205  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:02.134404  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 132363us
I20260709 13:00:02.135174  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.138854  8898 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:00:02.133805  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 110047us
W20260709 13:00:02.140872  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79109.7us
I20260709 13:00:02.141686  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.143635  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:02.144651 10256 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.145619 10229 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.145985 10229 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:02.146723 10229 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.146961  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.147764 10229 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.150043  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.151770  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.154088  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.155467  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.156265 10189 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:02.143708 10194 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.157542  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.158347 10180 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.148072 10208 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.160228 10208 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.160585 10208 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.162220 10243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f573b86fda4b3e8fd982da9615dc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:02.166280 10201 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.166821 10256 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.167275 10256 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.167809 10256 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.168470 10189 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.172398  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.156762 10197 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.182395 10171 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.184369 10232 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.169090 10217 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:00:02.185882  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.187280 10257 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.187510 10230 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.187893 10230 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.189091  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.189843 10257 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.192071  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.193011 10258 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.173336 10223 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.193251  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.194371 10255 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.195803  8817 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.196384 10243 tablet_copy_client.cc:806] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:02.196810  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.166816 10201 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.196998 10243 tablet_copy_client.cc:670] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:02.197853 10201 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.199520  8896 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:02.200307 10224 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.201787 10201 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.170270 10189 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.171154 10249 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.178242 10245 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.178743 10226 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.179466 10182 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.212705 10245 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.213127 10182 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.213768 10245 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.213968 10182 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.215013 10182 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:02.149148 10218 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:02.215149 10245 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.217569 10218 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.217978 10218 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.212682  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.190243  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.224440  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.204262  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.207424  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.229787 10194 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.230224 10194 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.230584 10194 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:02.235869  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45689us
I20260709 13:00:02.238543  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.232833  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.233664 10180 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.239527 10221 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.188923 10230 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.244473 10230 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.243580 10263 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.231012 10243 tablet_copy_client.cc:538] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:02.241506  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.252581  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.259140  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:02.256026  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.255520  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.260812  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.262305  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.263763  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.264045  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.264921  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.265089  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.271788  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.272701  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.273908  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.278215  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.278566  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.300621 10261 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.255755 10258 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.283365  9447 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:02.284296  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.306994  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.276863 10180 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.261677 10226 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.278299 10209 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.280210 10185 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.279057 10188 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.284711 10217 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.261678 10263 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.279815 10178 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.318819 10178 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.262741 10221 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.319235 10178 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.282078 10191 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:00:02.299934 10208 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.300259 10194 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.256125 10262 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.326939 10262 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.327358 10262 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.304821  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.305738 10177 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.277310 10249 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.255762 10254 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.308827  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.311661  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.282401 10206 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.279625 10200 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.333580  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.311160 10216 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.336781  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.309454 10229 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.338179  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.311867 10223 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.335971 10194 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.339972 10177 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.341648 10180 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.327827 10218 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.364948 10194 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:02.352658 10262 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.372254 10194 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.372659 10194 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.361893 10241 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:02.373520 10194 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.374497 10241 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.374868 10241 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.356601  8748 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.374877 10194 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.375958 10241 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.376943  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.344964 10199 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.345101  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.345546  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.390012  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.390409 10243 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 13:00:02.391232  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.391882  8749 raft_consensus.cc:2376] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2. Re-sending same reply.
I20260709 13:00:02.346556  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.346876  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.347224  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.347322  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.398285  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.347463  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.347888  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.400121  8820 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.348376  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.402539 10193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:45262 (request call id 10) took 2050 ms. Trace:
I20260709 13:00:02.403041 10193 rpcz_store.cc:276] 0709 13:00:00.355478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:00.355621 (+   143us) service_pool.cc:224] Handling call
0709 13:00:00.421257 (+ 65636us) log.cc:844] Serialized 506 byte log entry
0709 13:00:01.116201 (+694944us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b1800045b20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f6428b0b 00007f90f64adb63 00007f90f64d5752 00007f90fb495d29
0709 13:00:02.401203 (+1285002us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":449750,"raft.queue_time_us":504030,"raft.run_cpu_time_us":10465,"raft.run_wall_time_us":104539,"spinlock_wait_cycles":40485248,"thread_start_us":185851,"threads_started":1,"wal-append.queue_time_us":232981,"wal-append.run_cpu_time_us":3616,"wal-append.run_wall_time_us":1193425}
I20260709 13:00:02.406486 10231 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.405860 10180 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:02.407397  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.408234 10194 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.408566 10194 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.409341 10194 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.411141  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:02.411979 10180 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.412420 10180 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.333078 10208 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.348610  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:02.336352 10254 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:02.343369 10200 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.413734  8743 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:02.413759 10254 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.413311 10180 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.379527  8889 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.338866 10183 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.348286 10178 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.385038 10256 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.377177 10241 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.344352  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:02.426156  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 81856.2us
I20260709 13:00:02.427175  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.355170 10216 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.386785 10218 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.429137  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.388077 10262 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.414593  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.343189  8817 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.445230 10217 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.445746  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.448179  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.451997  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:02.419523  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.419560  8823 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:02.443773  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.444167  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.454305  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.455379  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.458482  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:02.463629  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 157316us
I20260709 13:00:02.444397  8748 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.463924  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:02.441594 10201 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.465967 10269 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.451186 10226 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.424724 10197 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:00:02.505441  8745 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.456764 10199 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.456377 10263 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.457880 10217 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.508410  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.464615 10216 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.469511 10218 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:02.528018  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.503048  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.567263  8741 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:02.574054 10268 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:00:02.550875 10202 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.628707  8819 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:00:02.642226  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72552.6us
I20260709 13:00:02.644522  8741 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:02.648397  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 228932us
I20260709 13:00:02.650319  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.629681 10202 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:02.718377 10178 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.723244 10194 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:02.737430  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 89947.1us
I20260709 13:00:02.742147 10260 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:02.743539 10260 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.743947 10260 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.746804  8743 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:02.761620  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 111339us
I20260709 13:00:02.649832 10263 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:00:02.774885  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.776038  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.774165 10178 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.783869 10269 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.794611  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.802655  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.803761  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.804826  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.795343 10275 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.805270  8745 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:02.805770 10275 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.806432 10275 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.807116 10275 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.774667 10256 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.798840 10244 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.807710 10245 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:02.808495 10244 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.809378 10244 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.809496 10224 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:02.811298 10224 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.811739 10224 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:02.812799 10224 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.814777  8647 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:02.815725 10224 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.824260  8819 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:02.857241  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52471.5us
I20260709 13:00:02.858194  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:02.858580  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.836261  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.745024 10260 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.834601 10269 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.865435  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:02.870122 10182 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:02.867426 10177 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.791131 10254 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.898187  8820 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.898631  9447 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:02.906451  8894 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:02.871591 10223 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:00:02.882716 10182 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:02.893764  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:02.898466 10216 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.919250 10275 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:02.914729 10182 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.912708 10217 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.928537 10216 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:02.936568  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67752.8us
W20260709 13:00:02.936651  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67503.5us
I20260709 13:00:02.938359  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:02.945541  8747 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.946122  8744 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.945912  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:02.954212  8743 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.957346  9441 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.950804 10239 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:00:02.967535  8822 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:02.949963 10275 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:02.974138 10241 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.530s	user 0.002s	sys 0.010s
I20260709 13:00:02.981714  8817 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:02.985978 10254 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:02.985595  8816 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:02.986881 10264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.017s	sys 0.020s
W20260709 13:00:02.987311 10264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.017s	sys 0.020s
I20260709 13:00:02.977357 10262 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:00:02.975931 10189 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:00:02.989909 10262 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.990020 10189 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.990325 10262 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.990509 10189 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:02.991448 10189 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.992731 10189 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.991384 10262 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:02.999930 10216 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.019007  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.020838  8817 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.013231 10202 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.023859 10188 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:03.036486  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63974.2us
I20260709 13:00:03.036556 10182 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.097141  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:03.097859  9448 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 2: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:03.098266  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:03.099157  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:03.041041 10182 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.090005 10262 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.090853 10188 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:03.107205  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:03.102982 10193 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:03.095145 10275 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:00:03.090022 10254 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:00:03.101042 10238 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:03.044763 10180 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:03.113750 10182 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:03.120980 10238 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.123809 10238 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.124848 10238 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:03.117491 10182 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:03.126902  8743 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:03.127020 10182 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.128729 10182 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.112416 10188 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.112738 10193 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:03.150440 10193 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.160353 10230 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.137092 10188 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.160970 10230 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.189863  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.193316 10224 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.193928 10224 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.190932  8824 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:03.219487  8644 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.204811 10229 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:03.234230  8646 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 194) took 1456 ms. Trace:
I20260709 13:00:03.245193  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.250114  9448 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:03.249915  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.260807  8821 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:03.268029 10292 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:00:03.270155  8891 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.237371  8646 rpcz_store.cc:276] 0709 13:00:01.761656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:01.761790 (+   134us) service_pool.cc:224] Handling call
0709 13:00:03.217766 (+1455976us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:02.089781 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:02.092884 (+  3103us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:02.092922 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:00:02.092940 (+    18us) tablet.cc:662] Decoding operations
0709 13:00:02.093265 (+   325us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:02.093339 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:02.093369 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:02.093535 (+   166us) tablet.cc:701] Row locks acquired
0709 13:00:02.093553 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:00:02.093690 (+   137us) write_op.cc:270] Start()
0709 13:00:02.093801 (+   111us) write_op.cc:276] Timestamp: P: 1783602002093650 usec, L: 0
0709 13:00:02.093823 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:00:02.095330 (+  1507us) log.cc:844] Serialized 536 byte log entry
0709 13:00:02.723948 (+628618us) op_driver.cc:464] REPLICATION: finished
0709 13:00:02.928617 (+204669us) write_op.cc:301] APPLY: starting
0709 13:00:02.928699 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:02.928793 (+    94us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:02.928913 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:02.930097 (+  1184us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:02.930967 (+   870us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:02.931020 (+    53us) write_op.cc:312] APPLY: finished
0709 13:00:02.931208 (+   188us) log.cc:844] Serialized 23 byte log entry
0709 13:00:02.931350 (+   142us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.931421 (+    71us) write_op.cc:454] Released schema lock
0709 13:00:02.931456 (+    35us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:02.931828 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:02.931856 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":204425,"apply.run_cpu_time_us":1570,"apply.run_wall_time_us":3196,"num_ops":1,"prepare.queue_time_us":223552,"prepare.run_cpu_time_us":47435,"prepare.run_wall_time_us":179130,"raft.queue_time_us":365144,"raft.run_cpu_time_us":94418,"raft.run_wall_time_us":190660,"replication_time_us":629969,"thread_start_us":1163981,"threads_started":4,"wal-append.queue_time_us":192587}]]}
I20260709 13:00:03.242055 10229 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:03.388877  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:03.391273  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.287638 10188 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.421124 10285 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.421650 10285 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.423807 10285 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.413210 10230 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:03.394938 10280 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.001s
W20260709 13:00:03.434689  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 192663us
W20260709 13:00:03.451874  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.417752 10188 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.460016 10188 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.461256  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.438027 10224 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.461838 10188 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.474319  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:03.479956  8788 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 31c225a50048451db8964738c5db7540. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:03.489518  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.490510  9447 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:03.468011 10292 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:03.492980  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 390143us
I20260709 13:00:03.494947  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.495878  8824 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:03.483301 10224 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.481627 10188 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.503820  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:03.499150 10224 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.497781 10239 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.483202 10258 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:00:03.520004 10260 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.509486 10285 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.514765 10224 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.527644 10285 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.541079 10308 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.541682 10308 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.544684 10308 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.545308 10193 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:03.517069 10239 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.546042 10188 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.545912 10193 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.547940 10239 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.548184 10193 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.547940  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.550256  8740 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.501164 10292 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:03.555274  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.533905 10301 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:03.539847 10285 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:03.645654 10299 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:00:03.647414 10305 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:00:03.647462 10188 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.648325 10188 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.645871 10292 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.652269 10188 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:03.641578 10274 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:00:03.604074 10193 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.637084 10301 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.708739 10180 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.705209 10217 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.726341 10258 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.728716 10258 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.732281 10258 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.721899 10193 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.736714 10193 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.740674 10301 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.725792 10180 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.750164 10180 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.737946 10217 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.757200 10217 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:03.716460 10182 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 13:00:03.769210 10224 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.772567 10224 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.793859  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.789862 10194 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.795557  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.796101 10194 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:03.797169  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.797927  9447 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:03.799003  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.799806  9440 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:03.791221 10293 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.801684 10293 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.804335 10293 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:03.791669 10297 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:03.797096 10194 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.805099 10297 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.807546 10297 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:03.810161  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.811007  8743 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.812191  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.812944  8748 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.793064 10285 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.816248  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.816995  8817 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
W20260709 13:00:03.696555  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 159050us
I20260709 13:00:03.818639 10285 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.819097  8816 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.819636  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:03.819995  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.820258  8815 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:03.820837  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.390075  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:03.827000 10182 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.839330  8647 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 281) took 1371 ms. Trace:
I20260709 13:00:03.846006  9440 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.844398  8647 rpcz_store.cc:276] 0709 13:00:02.468431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.468576 (+   145us) service_pool.cc:224] Handling call
0709 13:00:03.839272 (+1370696us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:03.016381 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:03.018271 (+  1890us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:03.018325 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:00:03.018352 (+    27us) tablet.cc:662] Decoding operations
0709 13:00:03.019910 (+  1558us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:03.027119 (+  7209us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:03.027215 (+    96us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:03.027475 (+   260us) tablet.cc:701] Row locks acquired
0709 13:00:03.027500 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:00:03.027696 (+   196us) write_op.cc:270] Start()
0709 13:00:03.027846 (+   150us) write_op.cc:276] Timestamp: P: 1783602003027644 usec, L: 0
0709 13:00:03.027875 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:00:03.028733 (+   858us) log.cc:844] Serialized 536 byte log entry
0709 13:00:03.177759 (+149026us) op_driver.cc:464] REPLICATION: finished
0709 13:00:03.178279 (+   520us) write_op.cc:301] APPLY: starting
0709 13:00:03.178382 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:03.178518 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:03.178647 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:03.178888 (+   241us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:03.179087 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:03.179185 (+    98us) write_op.cc:312] APPLY: finished
0709 13:00:03.179410 (+   225us) log.cc:844] Serialized 23 byte log entry
0709 13:00:03.179587 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:03.179692 (+   105us) write_op.cc:454] Released schema lock
0709 13:00:03.179737 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:03.180090 (+   353us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:03.180114 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43333,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":288,"apply.run_cpu_time_us":1780,"apply.run_wall_time_us":1780,"num_ops":1,"prepare.queue_time_us":159214,"prepare.run_cpu_time_us":3848,"prepare.run_wall_time_us":12703,"replication_time_us":149703,"thread_start_us":112416,"threads_started":1}]]}
I20260709 13:00:03.833323 10188 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:03.849539 10188 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.850455  9447 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:03.850946  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.834059 10193 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:03.853420  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:03.853267 10193 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.853615 10177 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:03.854007  9442 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.855371 10193 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.858009  9440 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:03.858278  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:03.859179 10249 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.835274  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.860040 10243 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:00:03.860056  8748 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:03.851267  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.861078 10243 tablet_bootstrap.cc:492] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 13:00:03.861379  9444 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:03.861609 10260 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.861836 10243 ts_tablet_manager.cc:1403] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 1.472s	user 0.032s	sys 0.033s
I20260709 13:00:03.837770  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.851544  9448 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:03.863004  8743 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:03.851599  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.863602  9442 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:03.852149  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.852540  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.864714  9446 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:03.864866  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1064) took 1031 ms. Trace:
I20260709 13:00:03.858455 10201 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.862154 10260 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.835055 10224 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:03.865250  8820 rpcz_store.cc:276] 0709 13:00:02.833182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.833543 (+   361us) service_pool.cc:224] Handling call
0709 13:00:02.857582 (+ 24039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:02.898557 (+ 40975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:03.864667 (+966110us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b54000811d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:03.864811 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92994304}
I20260709 13:00:03.865921 10224 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.851929  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:03.867278  9441 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:03.868048 10224 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:03.868404 10260 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.864251  9449 raft_consensus.cc:2376] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3. Re-sending same reply.
I20260709 13:00:03.870507 10260 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.871603 10260 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:03.871965 10260 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:03.865536 10201 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.875496 10201 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.878540 10256 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.878569 10285 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.879097 10256 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:03.879334 10285 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.880131 10256 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.884955  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.885643  8892 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:03.885993  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.886636  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.886837  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.887281  8889 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.888444 10200 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.888834 10200 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.889153  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.889457 10208 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:03.889833 10208 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.890158 10208 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.893590  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.894199  8820 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:03.889165 10200 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.895098  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.896044  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:03.896637  8815 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.896579  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.897367  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.898361  8817 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:03.898455 10182 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.899773  8816 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:03.900563  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.901752  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:03.903614 10194 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.903678 10189 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:03.904680 10189 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.905766 10235 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.903801 10180 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:03.906291 10235 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.907008 10180 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.881193 10256 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.909824 10230 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:03.910352 10230 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:03.911495  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.911257 10230 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.912705 10230 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.913071 10271 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:03.913619 10271 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.914858  8643 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 reported cstate change: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 1 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.916070  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.916595 10193 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:03.916728  8816 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.917090 10193 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.918627  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.919409  8816 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:03.921419  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:03.924764  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.934656 10229 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.936336 10244 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.937032 10285 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.937655 10271 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.938156  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.938709 10182 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.938163 10189 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.865453 10188 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.939922  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.940986  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.942188 10235 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.942274 10197 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:03.942725 10197 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:03.943078 10197 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.940917 10279 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:03.943642 10279 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.944048 10279 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.942355 10286 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.946116 10286 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.946513 10286 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.951979 10199 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:03.952657  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:03.952559 10199 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.953308  8745 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:03.944073 10261 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.954492 10193 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.954546 10199 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:03.954684 10261 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:03.956959 10261 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:03.957713  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.958416  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.978724  8824 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:03.957738  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:03.962651  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:03.979625  8817 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:03.954123  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:03.984258 10279 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.967970  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.990247  8897 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:03.991199  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.991758  8897 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:03.992072  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:03.992487  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.992650  8743 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.969251  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.994105  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.994323  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.995138  8891 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:03.995289  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.995901  8893 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:03.996045  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.996697  8891 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:03.996829  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.969988  8889 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:04.000018  8743 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:03.970579  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.001209  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:03.971524  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.001966 10292 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:04.002067  8896 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:04.002486 10292 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.003396 10292 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.004084  8893 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.004504 10208 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.972462  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.977265  8815 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:03.977835  8898 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:03.980942 10294 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.049638 10294 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.050113 10294 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.050684  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1312) took 1204 ms. Trace:
I20260709 13:00:04.051046  8898 rpcz_store.cc:276] 0709 13:00:02.846503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.858323 (+ 11820us) service_pool.cc:224] Handling call
0709 13:00:04.050511 (+1192188us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003bdb54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:04.050606 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537838848}
I20260709 13:00:04.052222 10208 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.054113 10200 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:03.982756  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:04.057103 10200 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:04.057687  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 74992.8us
I20260709 13:00:03.872830 10260 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.060019 10260 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.966528  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:03.986125 10261 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.986416 10244 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:03.988049  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:03.976831 10286 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.022856 10279 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.064893  8890 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.064405 10182 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.065770 10182 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:04.065840 10244 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:04.066290 10244 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:04.009609 10206 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.936401 10180 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:03.865042 10243 raft_consensus.cc:359] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.066627 10182 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.022792 10197 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.024871 10189 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.028689 10230 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:04.072623  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:04.030287  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:04.076203  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45985.1us
I20260709 13:00:04.076653  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:04.076936  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.069726 10206 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.030386  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.078538  9441 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:03.992506 10194 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.031497  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.032454  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.036464  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.037132  8748 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:04.037727 10256 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.038470  8740 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:04.043120  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:04.044508  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.044526  8741 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:04.045604  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.047205  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.063127  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.066355 10286 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.009703 10245 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.090698  8747 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:04.090756 10230 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.091368 10230 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:04.091106 10245 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.091630  8743 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.092193  8894 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:04.092234 10230 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.067844  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:04.093600 10245 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:04.064910  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 76936.2us
I20260709 13:00:04.067121  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.094619  8815 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:04.096194  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.070159 10243 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 13:00:04.097460 10243 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:04.097891  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.068871 10279 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.098335  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.098784 10279 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.098999  8815 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:04.099274 10279 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.100111 10243 ts_tablet_manager.cc:1434] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.238s	user 0.003s	sys 0.006s
I20260709 13:00:04.100628  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.064393 10261 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.101423 10201 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.101476 10245 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.029217  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.101874 10201 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.102111  8759 tablet_copy_service.cc:342] P b60591217ee24378bb66a86d6d0dda52: Request end of tablet copy session 112105225cc140459688be5d31dcb071-d4f573b86fda4b3e8fd982da9615dc8e received from {username='slave'} at 127.0.0.1:45274
I20260709 13:00:04.080770 10260 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:04.102818  8759 tablet_copy_service.cc:434] P b60591217ee24378bb66a86d6d0dda52: ending tablet copy session 112105225cc140459688be5d31dcb071-d4f573b86fda4b3e8fd982da9615dc8e on tablet d4f573b86fda4b3e8fd982da9615dc8e with peer 112105225cc140459688be5d31dcb071
I20260709 13:00:04.102896 10226 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.103343 10226 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.103574 10226 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.103953  8746 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
W20260709 13:00:04.104650  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.097074  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:04.101491 10261 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.105615  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.067276 10244 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.081414  8892 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.093616  8749 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.081897 10206 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.083191 10294 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.085845  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:04.086755  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 54377.8us
I20260709 13:00:04.084501 10202 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.088755  8745 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:04.085391 10194 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:04.088900 10256 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.134047 10194 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.088783 10292 raft_consensus.cc:2764] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:00:04.074366 10189 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.102298 10201 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.081017 10314 ts_tablet_manager.cc:933] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.136029 10279 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.107694  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:04.139120  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53468.4us
I20260709 13:00:04.144351  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.144431  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:04.112962 10286 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.138206  8744 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:04.106336  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.102049 10245 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.117316  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.117583  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.119639  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.131341 10260 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:04.149396 10249 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.120354  8749 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:04.125911  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.150020 10260 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.145210 10286 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.150519 10260 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.146977 10194 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.151515 10194 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.151063 10286 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.149993  8891 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.127660 10197 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.146567 10188 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.128439  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.121371 10244 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.132534 10202 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.141188  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.138938 10189 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.135416 10256 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.147048  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.125413 10200 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.155936  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.167529  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.154636 10208 raft_consensus.cc:2764] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:04.171423  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.171880 10208 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.172284 10208 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:04.172614  8817 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:04.172696 10208 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.173168  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.166955 10202 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.174851  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.139284 10206 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.151850 10194 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.170070  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.179184  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.180598  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:04.180923  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.176656  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.177769  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.147948 10245 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.181952  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.145749  8818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:04.182399  8747 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:04.178484  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.154982 10199 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:04.183591 10231 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.184016 10294 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.184105  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.184880  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.185983  8787 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:04.187168  9446 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:04.187400  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.188128  9445 raft_consensus.cc:1217] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:04.204929  8817 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.205847  9441 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.206187  9445 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:04.208254  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.189263  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.208963  9449 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.154246 10249 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.191715  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.220053  8898 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:04.169404 10295 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.185947 10279 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.192759  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.284458  8892 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.153680 10188 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:04.194248  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.186146 10285 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.249658 10289 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.194869 10231 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:04.198649  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.194258  8745 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:04.184247  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.304260  9443 raft_consensus.cc:2368] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.304431  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.202235 10271 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:04.210125  8789 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:04.306324  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 96256.5us
I20260709 13:00:04.309229  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.309810  8822 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.167023 10182 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:00:04.186671 10230 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.190344  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.269869  8713 leader_election.cc:400] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.196842  8818 raft_consensus.cc:606] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.282922  9449 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
W20260709 13:00:04.287045  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 110674us
I20260709 13:00:04.289799  9442 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.197100  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.259680 10183 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.258677 10199 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.331130  9447 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:04.292193 10285 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:04.266142 10256 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.243075  8816 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:04.299741 10231 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.312908 10200 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.346305 10200 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.313134  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:04.204849 10208 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.329263  8818 raft_consensus.cc:614] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:04.355444 10254 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.356112 10254 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:04.356469 10208 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.287853 10188 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.357195 10254 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.320704  8822 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
W20260709 13:00:04.327622  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 148763us
W20260709 13:00:04.315497  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 56141.2us
I20260709 13:00:04.287693 10224 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.298607  8892 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:04.361428  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:04.363694  8743 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:04.364012  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.348626 10260 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.265972 10249 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.301160 10289 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.337960 10189 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.338641  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.334556 10183 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:04.400851 10249 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.400516 10183 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.334725 10199 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.315764 10216 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.314859 10177 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.336117 10285 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.372182  8713 leader_election.cc:400] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
W20260709 13:00:04.330252  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 134716us
I20260709 13:00:04.421273  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.372625 10193 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.417553 10188 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.429698 10201 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.359426 10256 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.361181 10178 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.436008 10231 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.374727 10230 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:04.437602 10230 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.313760 10226 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.384341  8749 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:04.438939 10244 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.439728 10230 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.384066 10289 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.440398 10226 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.370131 10314 tablet_copy_client.cc:323] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 13:00:04.372514 10294 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.383534  9442 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.417492  9443 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:04.399583  8893 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.427563 10177 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.446161 10294 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.446979 10177 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.447118 10294 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.420571 10216 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.451191 10216 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.429382  8713 leader_election.cc:403] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:04.453325  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 81294.3us
I20260709 13:00:04.454169  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.454491 10231 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.429828 10193 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.431516  8744 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:04.371119  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.369671 10224 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.434228  8894 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:04.435205 10188 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.436286 10256 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.436496 10260 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.436801 10178 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.358635 10180 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:00:04.439481 10244 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.443256  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.460130 10229 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:04.428221 10321 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:00:04.457890  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.458722  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.455622 10325 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:00:04.468681  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 97651.1us
I20260709 13:00:04.478929  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.452785 10202 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.475064 10261 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.488857 10261 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.490124  8890 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:04.490629  8889 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:04.491423  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.491734  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.492479  9446 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:04.495061 10244 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.495854 10271 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.496599 10245 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.498087 10271 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:04.498561 10271 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.500515 10271 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.501793 10245 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.475217  9449 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.443648  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.485591 10178 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.486450 10249 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:04.461172  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128279us
I20260709 13:00:04.502372  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:04.469723 10216 raft_consensus.cc:476] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:00:04.505602 10260 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:04.510663 10260 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.488870 10202 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:04.513234  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 92015.8us
I20260709 13:00:04.471390 10329 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.517154  9449 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:04.476318  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.517652  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.478422  8818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:04.521802 10330 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.525059  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.526309 10193 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.482218 10188 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.526929  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.527061 10188 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:04.483403 10229 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:04.528282 10224 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.528057 10188 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.528188 10229 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.529022  9442 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.529784  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.529897 10189 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.530332 10189 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.530454 10249 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.530910 10249 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.531220 10249 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.508945 10200 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.532491 10201 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.531677  8818 raft_consensus.cc:606] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.535524  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.532387  8893 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:04.537103  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.537917  8898 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:04.537176  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.537820 10231 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.538983  8891 raft_consensus.cc:2368] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
I20260709 13:00:04.540973  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.534754  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.545498  8821 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:04.542665  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.542902  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.543287  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.539435  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:04.547001  8820 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.540321  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.544816  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.549778  8898 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.552278  8823 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:04.549778  8892 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:04.558007  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.558720  9449 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:04.559510  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 31c225a50048451db8964738c5db7540 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 13:00:04.560192  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 31c225a50048451db8964738c5db7540 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-31c225a50048451db8964738c5db7540
I20260709 13:00:04.562696  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.539634  8818 raft_consensus.cc:614] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:04.517320 10194 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.546130 10277 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:04.571547  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 216766us
I20260709 13:00:04.545469 10201 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.576936  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.577534  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.584383  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.565785 10286 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.579061  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.588471 10199 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.595415 10200 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.572417  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.601588  8743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:32798
I20260709 13:00:04.602427  8743 raft_consensus.cc:606] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.602754  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.604249 10330 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.604629 10330 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.604940 10330 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.567221 10231 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.605984  8823 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3.
I20260709 13:00:04.606732  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.610687  8892 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.619114  8820 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:04.620626  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.622664  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.574067 10183 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.625352  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 13:00:04.628438  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 149604us
I20260709 13:00:04.629485  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:04.631390  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:04.570840 10245 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:00:04.588615  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.637667  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.601655 10333 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.638839  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.639886  8821 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:04.638759  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.645112  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.607297 10277 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.646534  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.624320 10271 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:04.650345  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.624603 10286 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:04.635928 10202 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:04.660908 10254 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.672508 10330 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.606343 10201 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.644052  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.645991 10216 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:04.683820  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.684643  8820 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:04.685765  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49672us
I20260709 13:00:04.694541  8712 leader_election.cc:400] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.694896  8712 leader_election.cc:403] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:04.696072  8712 leader_election.cc:400] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
I20260709 13:00:04.696380  8712 leader_election.cc:403] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:04.639765 10183 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.659358 10329 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.702486  8897 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:04.673918 10194 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:04.648142 10333 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.687395 10202 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:04.699150 10178 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.712502  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:04.716507  8820 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.700311 10260 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.674847 10271 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.705420 10329 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.687923 10289 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.722481  8892 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.708259 10325 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.708407 10336 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.727676  8892 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:04.720767 10329 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.716281 10178 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.715251  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:04.718567 10200 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.736382  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:04.739918  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.728883 10343 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.724707 10189 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.709019 10231 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.744839 10183 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.737426 10178 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.718276 10277 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.752202  8644 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 297) took 2506 ms. Trace:
I20260709 13:00:04.738015 10254 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:04.744912 10337 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:00:04.720804 10286 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.758167  8822 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:04.734381 10336 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.749606 10202 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.755012 10285 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.770576  9445 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:00:04.792505  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 153520us
I20260709 13:00:04.795002  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.795571  8892 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.799268  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.799921  8820 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:04.756644  8644 rpcz_store.cc:276] 0709 13:00:02.242360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.244162 (+  1802us) service_pool.cc:224] Handling call
0709 13:00:02.259409 (+ 15247us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:00:04.749144 (+2489735us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:03.250528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:03.875868 (+625340us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b5c000b600c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc6c1331 00007f90fc6c1221 00007f90fc6bbf6c 00007f90f9f19d69 00007f90f9f050a6 00007f90f9f04d58 00007f90f9f099f1 00007f90f9f099a9 00007f90f9f09939 00007f90f9f09901 00007f90f9f08b2d
0709 13:00:03.875951 (+    83us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:03.875993 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:00:03.876014 (+    21us) tablet.cc:662] Decoding operations
0709 13:00:03.876400 (+   386us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:03.876457 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:03.876493 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:03.876679 (+   186us) tablet.cc:701] Row locks acquired
0709 13:00:03.876699 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:00:03.876840 (+   141us) write_op.cc:270] Start()
0709 13:00:03.877089 (+   249us) write_op.cc:276] Timestamp: P: 1783602003876796 usec, L: 0
0709 13:00:03.877114 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:03.959406 (+ 82292us) log.cc:844] Serialized 536 byte log entry
0709 13:00:04.193404 (+233998us) op_driver.cc:464] REPLICATION: finished
0709 13:00:04.438596 (+245192us) write_op.cc:301] APPLY: starting
0709 13:00:04.438688 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:04.440767 (+  2079us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:04.440791 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:04.441043 (+   252us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:04.441181 (+   138us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:04.441233 (+    52us) write_op.cc:312] APPLY: finished
0709 13:00:04.445469 (+  4236us) log.cc:844] Serialized 23 byte log entry
0709 13:00:04.445627 (+   158us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:04.445698 (+    71us) write_op.cc:454] Released schema lock
0709 13:00:04.445739 (+    41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":227433,"reported_tablets":4,"spinlock_wait_cycles":19730304,"child_traces":[["op",{"apply.queue_time_us":241417,"apply.run_cpu_time_us":1681,"apply.run_wall_time_us":7453,"num_ops":1,"prepare.queue_time_us":282,"prepare.run_cpu_time_us":18097,"prepare.run_wall_time_us":735902,"replication_time_us":299513,"spinlock_wait_cycles":5171072,"thread_start_us":202188,"threads_started":1}]]}
I20260709 13:00:04.806285  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.806488  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.806877  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:04.807993  8892 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:04.788841 10201 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.814810  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.815174 10336 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.815495  8890 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:04.816457  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.802450 10189 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.790807 10330 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.813848  8834 tablet_copy_source_session.cc:215] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:04.793308 10260 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:04.802485 10333 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.802755 10271 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.850850 10201 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.862862  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.863394  8892 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.858458 10289 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.858848  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:04.866866  8890 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:04.867867 10329 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.869789 10343 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
I20260709 13:00:04.872854  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:04.873207 10178 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.874332  8713 leader_election.cc:400] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.876116  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:04.877453  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:04.877590 10178 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.878093  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.878897 10216 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.855537 10260 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:04.880676  8895 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:04.881668 10260 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:04.862803 10201 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:04.882603 10201 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.879845 10216 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:04.886109 10285 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:04.886322 10216 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:04.887538 10216 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.889705  8748 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:04.878871  8818 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:04.891474 10295 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:04.905085  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.907049  8895 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:04.907310  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:04.848080 10254 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:04.925930 10289 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:04.925007  8892 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:04.932307 10322 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.641s	user 0.003s	sys 0.008s
I20260709 13:00:04.927389  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:04.948521 10301 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:04.918015 10330 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.957991  8896 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:04.971793 10200 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3.
I20260709 13:00:04.973014  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:04.978802  8643 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 267) took 1947 ms. Trace:
I20260709 13:00:04.981561  8647 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:04.986049  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:05.017808  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:05.018231  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.018435  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.019375  8744 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:05.019280  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.018414  8896 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.020772  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.021268  8741 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.021191  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.022617  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.022868  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.029235  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.030058  8744 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.029248  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.031204  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.031368  8749 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:05.040292  8896 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:05.044492  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.057480  8742 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.028796 10289 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.031674  8747 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:05.059569  8896 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:05.059654  9447 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:04.966193 10249 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.098629  8898 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.100512  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.101135  8744 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:05.101944  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.103025  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.017369 10329 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.103770  8744 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.128719  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:05.130095  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.130695  8820 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:05.105616  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.131560  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:05.124514  8742 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.133047  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.133584  8742 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.135299  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:05.124480  8741 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:05.136620  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.138286 10271 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.138800 10271 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.139174 10271 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.139901  8741 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.141269  8741 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.136878  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.150128 10178 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.150748 10178 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.151239 10178 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.127226  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.157925  8863 leader_election.cc:400] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:05.158453  8863 leader_election.cc:403] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:05.159725  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.162137  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.163180  8740 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.058660  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:04.979189  8643 rpcz_store.cc:276] 0709 13:00:03.031359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.031928 (+   569us) service_pool.cc:224] Handling call
0709 13:00:04.978747 (+1946819us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:03.977809 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:03.978555 (+   746us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:03.978603 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:00:03.978630 (+    27us) tablet.cc:662] Decoding operations
0709 13:00:03.979823 (+  1193us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:03.979863 (+    40us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:03.979904 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:03.980069 (+   165us) tablet.cc:701] Row locks acquired
0709 13:00:03.980085 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:00:03.980201 (+   116us) write_op.cc:270] Start()
0709 13:00:03.980305 (+   104us) write_op.cc:276] Timestamp: P: 1783602003980163 usec, L: 0
0709 13:00:03.980325 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:00:03.980904 (+   579us) log.cc:844] Serialized 599 byte log entry
0709 13:00:04.454386 (+473482us) op_driver.cc:464] REPLICATION: finished
0709 13:00:04.455582 (+  1196us) write_op.cc:301] APPLY: starting
0709 13:00:04.455660 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:04.455757 (+    97us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:04.474562 (+ 18805us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:04.477874 (+  3312us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:04.478032 (+   158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:04.478084 (+    52us) write_op.cc:312] APPLY: finished
0709 13:00:04.478264 (+   180us) log.cc:844] Serialized 23 byte log entry
0709 13:00:04.484335 (+  6071us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:04.484429 (+    94us) write_op.cc:454] Released schema lock
0709 13:00:04.484470 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:04.484801 (+   331us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:04.484832 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":452409,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":1005,"apply.run_cpu_time_us":1877,"apply.run_wall_time_us":29165,"num_ops":1,"prepare.queue_time_us":35095,"prepare.run_cpu_time_us":2524,"prepare.run_wall_time_us":3334,"replication_time_us":473959}]]}
I20260709 13:00:05.174605  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.187089  8740 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:05.124944  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.156862 10343 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.084261 10289 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.158034 10245 raft_consensus.cc:2764] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:05.163211 10260 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:05.195168  8741 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.186246 10352 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:00:05.185189  8743 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:05.236835 10271 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.195786 10343 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.238559  8744 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.239629 10343 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.240188 10271 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.241420  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.241588 10289 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.191921 10339 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.242038  8823 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:05.242399 10339 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.242921 10339 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.244103  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:05.244408  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:05.244964  8895 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:05.203639 10202 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.258983  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:05.259757  8894 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:05.254590  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:05.186733 10254 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 13:00:05.274037  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114402us
I20260709 13:00:05.275066  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:05.339100  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64113.6us
I20260709 13:00:05.339977  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.340296  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:05.341029  8895 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:05.341952  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.345744  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.347615  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:05.394707  8823 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:05.264506  8889 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.341713 10188 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:05.342999 10189 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.341116 10245 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:05.454118  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 189866us
I20260709 13:00:05.343132 10343 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.350519 10260 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.265775 10182 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260709 13:00:05.343282 10208 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:05.253311 10332 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.143s
I20260709 13:00:05.459918  8742 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:05.450170 10336 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:05.468408  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.473601  8823 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:05.475804 10330 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.476365 10330 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:05.477258 10330 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.460840 10171 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:00:05.456988 10189 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:05.485129 10201 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 13:00:05.484771 10189 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.485556 10201 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.486017 10189 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.488283 10336 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.470119 10208 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.490567 10336 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:05.491345 10208 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:05.491887 10358 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:00:05.472079 10245 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.468320 10343 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.494664 10343 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.498852  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:05.499442  8897 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:05.468463 10249 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.499459  8898 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.466718 10188 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.502523 10188 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:05.503158  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.503822  8819 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:05.472965 10364 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.504602  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.505757  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.499815  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:05.509795  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.509894  8896 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:05.510887  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.499740  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:05.513507  8895 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:05.516754  9415 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3.
I20260709 13:00:05.519311  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:05.520021  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.522058  8740 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.523998  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.524750  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:05.526237  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 180538us
I20260709 13:00:05.531186  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:05.539083  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:05.473174 10260 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:05.526139 10361 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:00:05.546733  8816 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.521719 10364 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:05.543723 10334 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:05.549172  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.548998  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.523128 10177 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.520850  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.540652 10245 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:05.532377 10271 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:05.526394 10289 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:05.560434 10193 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.555575 10177 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:05.531188 10343 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:00:05.542712 10336 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.562744  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.551178 10352 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:05.548904 10201 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:05.568977  8749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.015s
W20260709 13:00:05.581169  8749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.000s	sys 0.016s
I20260709 13:00:05.562944  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:05.567456 10332 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:05.561597  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 209230us
I20260709 13:00:05.564910 10245 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:05.564679 10271 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:05.551066 10260 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.563999 10334 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:05.574496 10177 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.575320 10336 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.575353 10289 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:05.550825 10335 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 13:00:05.647765  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 101935us
I20260709 13:00:05.580107 10329 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:05.598840 10363 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:00:05.649161  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.558233 10364 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.655256  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:05.655459  9449 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.655861  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:05.582819 10216 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:05.651957 10245 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:05.662703 10360 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.404s	user 0.004s	sys 0.000s
W20260709 13:00:05.663347 10367 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:00:05.669193  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:05.648694 10193 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:05.689543 10193 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.690757 10289 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.649834 10332 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.666347 10336 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.653324 10271 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:05.682687  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:05.650848 10352 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:05.687994  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91995.9us
I20260709 13:00:05.695811 10256 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:05.699357  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 161) took 1085 ms. Trace:
I20260709 13:00:05.675737 10329 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.671130 10301 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:05.724073  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:05.724974  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1078) took 1182 ms. Trace:
I20260709 13:00:05.724260 10334 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.725389  8817 rpcz_store.cc:276] 0709 13:00:04.542102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.544726 (+  2624us) service_pool.cc:224] Handling call
0709 13:00:05.720210 (+1175484us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:05.724879 (+  4669us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12830464}
W20260709 13:00:05.720376 10178 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.002s
I20260709 13:00:05.696367 10332 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.726634 10178 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.747921  8815 raft_consensus.cc:2368] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:05.700696 10189 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:00:05.687191 10216 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:05.760389  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77799.6us
I20260709 13:00:05.763149  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:05.781620  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78709.4us
I20260709 13:00:05.705156 10229 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.702951 10256 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.719870 10329 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.832062 10279 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.841545  8896 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.778007 10335 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:05.841773 10279 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.760538 10178 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.842116 10335 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.845523 10335 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.846830 10256 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.847260 10229 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:05.704535 10378 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.705663  8745 rpcz_store.cc:276] 0709 13:00:04.614232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.624481 (+ 10249us) service_pool.cc:224] Handling call
0709 13:00:05.693858 (+1069377us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:05.698668 (+  4810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:05.698889 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:05.698917 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:05.698979 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:05.699031 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:05.699125 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:05.699283 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3211136}
I20260709 13:00:05.777782 10201 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:05.757694  8818 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:05.813573 10199 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.851771  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:05.850751 10314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31c225a50048451db8964738c5db7540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.876222  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:05.882810  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:05.885437 10178 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:05.865958 10229 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.854810 10202 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:05.890772  9448 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.877190 10199 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.721966 10339 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.886962 10349 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:00:05.894762  9448 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:05.892073 10342 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.729568 10301 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.896675  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:05.885993 10178 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.897816  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.899811  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:05.904769 10216 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:05.905400 10216 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.906205  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.906874  8821 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:05.884784 10336 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.908245  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:05.880335 10256 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.910185  8824 raft_consensus.cc:2368] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:05.872176  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:05.878185  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:05.916252  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:05.917302  8864 leader_election.cc:400] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:05.917667  8864 leader_election.cc:403] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:05.911912  9448 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.922473  9448 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.896638 10199 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.934082  8896 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:05.876999 10201 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.910640 10279 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.918480 10256 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.909616 10216 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.917310  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:05.917956 10260 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:05.909221 10336 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.928229  8893 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:05.901762 10202 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:05.948568  8743 raft_consensus.cc:2368] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:05.942896 10343 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.956262 10368 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:00:05.946768 10216 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:05.964788 10377 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:00:05.965906 10224 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.947091 10256 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.927336 10271 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:05.961731  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1076) took 1428 ms (client timeout 1593 ms). Trace:
I20260709 13:00:05.888610 10352 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:05.977872 10339 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:05.979858 10178 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:05.980921 10301 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:05.976275  8822 rpcz_store.cc:269] 0709 13:00:04.533503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.534544 (+  1041us) service_pool.cc:224] Handling call
0709 13:00:05.961521 (+1426977us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540033bdd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:05.961673 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487051008}
I20260709 13:00:05.968235 10343 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:05.974844 10216 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.981567 10245 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:05.963734  9448 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:06.014472  9448 pending_rounds.cc:85] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 4
W20260709 13:00:06.014540 10374 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.447s	user 0.002s	sys 0.000s
I20260709 13:00:05.993208 10343 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.990666 10364 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.026433 10177 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.027594 10197 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:06.028256 10329 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.001034  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:06.016119 10351 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:06.017187  9448 pending_rounds.cc:107] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:06.044040  9448 raft_consensus.cc:2889] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:06.052323 10201 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.053133 10271 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.029134 10177 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.028165 10197 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.165616  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:06.061707 10333 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.143931 10351 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:06.151460 10261 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:06.173101 10177 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.180040 10202 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.173609 10332 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.173676 10333 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.213750  8817 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
W20260709 13:00:06.224844  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 305244us
I20260709 13:00:06.182647 10261 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.226711  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:06.206447 10177 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.250526  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.294224  9448 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:06.228849  8896 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:06.244619  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 344822us
I20260709 13:00:06.361809 10224 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:06.277314 10182 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.365897 10386 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.366096 10182 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.366497 10386 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.367843 10277 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:06.369187 10182 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.379642 10333 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.381299  9443 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:06.389672 10202 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.392412 10202 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.394397  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:06.372216 10271 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:06.395460 10271 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:06.396294 10271 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.334324 10329 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.333841  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.402354  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:06.402809  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46233.2us
I20260709 13:00:06.404604  8897 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:06.245934 10245 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.406998  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:06.402011  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:06.402938  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:06.419167  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.018s	sys 0.025s
W20260709 13:00:06.419548  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.018s	sys 0.025s
I20260709 13:00:06.422118  8822 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:06.407800 10294 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.038s	sys 0.055s
I20260709 13:00:06.421348  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:06.424275  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:06.426811  8822 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.434396  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:06.444118  8822 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:06.446426 10261 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.407912 10224 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.416597 10339 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:06.425931 10294 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.039s	sys 0.055s
I20260709 13:00:06.397591 10189 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:00:06.456774 10382 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 13:00:06.458935  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.459672  8894 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:06.452282 10380 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.565s	user 0.006s	sys 0.007s
I20260709 13:00:06.408293 10199 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.434577  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:06.279034 10351 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.500469  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:06.501852  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:06.454521 10388 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:00:06.512159  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:06.506824  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.452616  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:06.450234 10197 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.515389  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:06.487715 10371 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.017s	sys 0.033s
I20260709 13:00:06.530361  8742 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:06.466007 10224 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.495807 10314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44010 (request call id 353) took 2508 ms. Trace:
I20260709 13:00:06.523944  8747 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:06.504314 10339 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.503845 10249 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.528824 10245 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.547801 10389 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.519320  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.545763  8894 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:06.534103 10177 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:06.544436 10371 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.018s	sys 0.034s
I20260709 13:00:06.537261  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:06.560659  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 333989us
I20260709 13:00:06.549027 10332 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.540109 10378 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:00:06.592756  8864 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.593s (SENT) [suppressed 10 similar messages]
I20260709 13:00:06.552094 10339 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.559528 10249 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.560566 10245 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.651401  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:06.630270  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:06.562983  8890 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.561692 10342 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:06.683316  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:06.684314  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:06.564805 10352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.033s	sys 0.033s
I20260709 13:00:06.539976 10314 rpcz_store.cc:276] 0709 13:00:03.958155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.958273 (+   118us) service_pool.cc:224] Handling call
0709 13:00:04.142635 (+184362us) ts_tablet_manager.cc:934] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
0709 13:00:05.521208 (+1378573us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:06.023714 (+502506us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:06.025011 (+  1297us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:06.457706 (+432695us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:06.458049 (+   343us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:06.458397 (+   348us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:06.466619 (+  8222us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":52402,"tablet-copy.queue_time_us":122136,"thread_start_us":274858,"threads_started":2}
W20260709 13:00:06.600438 10216 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 13:00:06.678086 10386 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.677673 10391 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.697461 10216 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.697893 10391 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.699649 10216 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.699874 10391 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:06.694525 10387 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:00:06.694900 10392 raft_consensus.cc:1064] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:00:06.695199 10249 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.627213 10197 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:06.689481 10371 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.702623 10197 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.703109 10371 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.568430 10208 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
W20260709 13:00:06.695955 10229 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.783s	user 0.003s	sys 0.008s
I20260709 13:00:06.695900 10245 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.697077 10314 tablet_copy_client.cc:806] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:06.632550 10199 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.693434 10224 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.682471  8895 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:06.564992 10389 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.704452 10392 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.705215 10371 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.717839 10391 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.707971 10229 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:06.705741 10352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.035s	sys 0.035s
I20260709 13:00:06.736155 10314 tablet_copy_client.cc:670] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
W20260709 13:00:06.712304 10343 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.000s
I20260709 13:00:06.637331 10368 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.744251 10391 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.751809 10343 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.755427 10389 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.756165 10391 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.757537 10343 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.759119 10368 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.761154 10339 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.646728  8744 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:06.637342  8745 raft_consensus.cc:2368] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:06.628536  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:06.796284  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 398038us
I20260709 13:00:06.665206  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:06.687269  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:06.709854 10197 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:06.710161 10256 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.002s
I20260709 13:00:06.846163 10256 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.841948 10391 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.842386 10389 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.849874  8746 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:06.851442 10256 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.852725 10229 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:06.852855  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 224253us
I20260709 13:00:06.858722 10391 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.864820 10339 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.866037 10339 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.866475  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.867213  8895 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.868649 10339 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:06.869096  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1082) took 2325 ms (client timeout 1993 ms). Trace:
I20260709 13:00:06.869442 10371 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.870299 10371 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.874140 10371 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.880267 10393 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.880841 10229 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:06.881464 10393 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.882043 10229 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:06.883112 10332 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:06.883715 10332 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.884207 10332 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:06.884773 10393 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.886562 10229 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:06.886816 10224 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:06.887893 10224 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.847787 10295 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:00:06.869663  8815 rpcz_store.cc:269] 0709 13:00:04.543134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.543975 (+   841us) service_pool.cc:224] Handling call
0709 13:00:05.514944 (+970969us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:06.840861 (+1325917us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400075154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:06.868907 (+ 28046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397431936}
I20260709 13:00:06.923552 10224 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.925072 10389 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.926251  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 216) took 1393 ms. Trace:
I20260709 13:00:06.926762  8820 rpcz_store.cc:276] 0709 13:00:05.532672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.532820 (+   148us) service_pool.cc:224] Handling call
0709 13:00:06.924777 (+1391957us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:06.925253 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:06.925668 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:06.925696 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:06.925771 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:06.925803 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.925860 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:06.925998 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17950080}
I20260709 13:00:06.928012 10389 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.928514  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:06.886819 10401 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:00:06.929912 10295 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:06.930624 10295 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:06.929976 10415 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:00:06.932171 10295 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.928061  8892 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:06.762931 10368 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:06.912421 10371 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:00:06.962738 10391 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:06.962873 10197 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.955125 10177 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:06.974807 10177 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:06.951751  8891 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
W20260709 13:00:06.973547  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 428716us
I20260709 13:00:06.981076  8742 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:06.984392  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 431489us
I20260709 13:00:07.000108  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:07.005467 10391 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.005880  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:07.006402 10391 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.985105  8896 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:07.008579  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.009310  8896 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:06.985776 10197 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:06.995530 10294 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.005244  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:06.984301  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:06.982275  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 359266us
I20260709 13:00:07.032161 10197 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.034940 10330 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.035825 10330 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:07.037474 10391 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.043735  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:07.073604  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57854.2us
I20260709 13:00:07.070705 10391 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.016146 10294 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.082441 10391 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.088485  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:07.089658  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47819.9us
I20260709 13:00:07.049467 10425 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:00:07.250051  8895 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:07.117622  8897 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.279798 10330 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.283669 10330 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.283900  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.284588 10330 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:07.276177  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125288us
I20260709 13:00:07.284992  8896 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:07.287273  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.292335 10425 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:07.293148 10425 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.292032  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:07.295830 10294 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:07.296113 10425 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.296900 10200 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:07.111244 10180 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.109897 10342 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.109897 10289 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.279755 10177 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:07.303785  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 303859us
I20260709 13:00:07.309368  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:07.106178 10271 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.109949 10392 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.117908 10256 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.218s
I20260709 13:00:07.336529  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.336998 10256 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:07.337459  8893 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:07.337689 10424 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.338652  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.339761 10391 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.339938  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.338580 10424 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:07.210644 10216 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.210162 10386 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.111255 10249 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.310737 10200 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:07.313410  8896 raft_consensus.cc:2368] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
W20260709 13:00:07.316329  8890 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.596s	user 0.014s	sys 0.000s
I20260709 13:00:07.111776 10206 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.346123 10206 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.346582 10200 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:07.348237  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1389) took 2095 ms (client timeout 1731 ms). Trace:
I20260709 13:00:07.348703 10206 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.317570 10180 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.323137  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:07.325479 10332 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:07.328063  8897 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:07.329231 10334 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.330344  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.330561 10335 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.331274  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.331663 10202 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.332242  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.333364  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.333626  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:07.333817  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.344781  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:07.337478 10256 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:07.360910  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 56199.8us
I20260709 13:00:07.350731 10295 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.364975 10342 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:07.365531 10334 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:07.348649  8746 rpcz_store.cc:269] 0709 13:00:05.252871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.254968 (+  2097us) service_pool.cc:224] Handling call
0709 13:00:06.394310 (+1139342us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aaf8c 00007f90f33aaef6 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:07.347968 (+953658us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400073ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:07.348142 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050191616}
I20260709 13:00:07.364266 10271 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.371843 10342 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:07.375736  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77726.7us
I20260709 13:00:07.358959  8890 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:07.343895 10386 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:07.381595 10249 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:07.404491 10391 raft_consensus.cc:2764] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:00:07.541518  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:07.415632  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.416860  8894 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:07.415663  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:08.271090  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44622.8us
I20260709 13:00:08.336035  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 243) took 1205 ms. Trace:
I20260709 13:00:07.680641  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.419329  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:07.671550  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:07.581420 10200 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.650739  8895 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:08.346963  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1743) took 1096 ms. Trace:
I20260709 13:00:07.542084  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:08.012866 10434 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:00:08.340598  8894 rpcz_store.cc:276] 0709 13:00:07.130256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.342331 (+212075us) service_pool.cc:224] Handling call
0709 13:00:07.361432 (+ 19101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.335748 (+974316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.335976 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:08.347920  8895 rpcz_store.cc:276] 0709 13:00:07.263822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.311885 (+ 48063us) service_pool.cc:224] Handling call
0709 13:00:08.346883 (+1034998us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:07.767416 10413 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.733s	user 0.001s	sys 0.012s
W20260709 13:00:08.290745 10295 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.920s	user 0.008s	sys 0.005s
I20260709 13:00:07.902308  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.336436 10202 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.626494 10286 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.582780 10330 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.581429 10326 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.469439  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1734) took 1029 ms. Trace:
I20260709 13:00:07.535812 10235 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.408576 10335 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.363735 10235 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.365810 10335 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.682932 10423 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:08.366776 10182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.010s	sys 0.011s
I20260709 13:00:08.367290  8890 rpcz_store.cc:276] 0709 13:00:06.414294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.465753 (+ 51459us) service_pool.cc:224] Handling call
0709 13:00:07.404974 (+939221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:07.544643 10178 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.752629 10334 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.387359 10289 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.290091  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.685403 10428 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.421233 10197 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:07.941663 10364 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.366693 10235 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:07.722721 10386 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:07.552778 10180 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:07.721958 10332 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:07.767953  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.417500 10432 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:07.748903 10425 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:07.721477 10189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36230 (request call id 12) took 4474 ms. Trace:
I20260709 13:00:07.671530 10424 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:08.369956  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 950537us
I20260709 13:00:08.371699  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1733) took 1959 ms. Trace:
W20260709 13:00:07.750548  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1090) took 2215 ms (client timeout 1766 ms). Trace:
I20260709 13:00:07.415788 10349 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.375900 10202 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.394335 10425 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:07.659147 10431 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.069854  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.362749 10326 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:08.368049 10182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.010s	sys 0.011s
I20260709 13:00:07.444883 10216 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.384788  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.381201  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.389963  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:08.386134  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 361900us
I20260709 13:00:08.380621 10335 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:08.387493 10386 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.395784  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:07.682632 10256 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.367311 10423 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.411378 10178 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.411391  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.412483  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.414328  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:08.371064 10200 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.395406 10249 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.393025 10197 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.382102 10286 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.421657 10335 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.462031 10378 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.469251 10249 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:08.469453  8817 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:08.470037  8823 raft_consensus.cc:2368] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
W20260709 13:00:08.396512  8743 rpcz_store.cc:269] 0709 13:00:05.535299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.540081 (+  4782us) service_pool.cc:224] Handling call
0709 13:00:07.750081 (+2210000us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54001c7d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:07.750345 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640398592}
I20260709 13:00:08.473465 10423 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.473583 10433 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.474229 10180 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:08.474699  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:08.467250 10235 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.475206 10178 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.405324 10432 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.476059 10182 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:08.476287  9443 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:08.391577 10189 rpcz_store.cc:276] 0709 13:00:02.968090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:02.968505 (+   415us) service_pool.cc:224] Handling call
0709 13:00:03.100890 (+132385us) log.cc:844] Serialized 506 byte log entry
0709 13:00:07.404515 (+4303625us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14721,"raft.queue_time_us":37353,"raft.run_cpu_time_us":1715,"raft.run_wall_time_us":2360,"thread_start_us":176790,"threads_started":1,"wal-append.queue_time_us":310781,"wal-append.run_cpu_time_us":2168,"wal-append.run_wall_time_us":1004054}
I20260709 13:00:08.476337 10424 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:08.496327 10433 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.477128 10256 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:08.497269 10249 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.498575 10235 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.499111 10433 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.395359  8889 rpcz_store.cc:276] 0709 13:00:06.412584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.416809 (+  4225us) service_pool.cc:224] Handling call
0709 13:00:06.452529 (+ 35720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:06.517528 (+ 64999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:06.521100 (+  3572us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:06.561995 (+ 40895us) log.cc:844] Serialized 26 byte log entry
0709 13:00:06.954007 (+392012us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:06.955068 (+  1061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.956191 (+  1123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:07.752350 (+796159us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:08.368128 (+615778us) raft_consensus.cc:1685] finished
0709 13:00:08.368158 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.371536 (+  3378us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":20155,"spinlock_wait_cycles":19464064,"thread_start_us":311954,"threads_started":1,"wal-append.queue_time_us":437350}
I20260709 13:00:08.477546 10349 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.479790 10216 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.391822 10428 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:08.468592  8789 connection.cc:256] RPC call timeout handler was delayed by -1.6090954799999508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
I20260709 13:00:08.398212 10364 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.419905 10431 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.493191  8820 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:08.484791 10335 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.494261  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:08.486826  8817 pending_rounds.cc:85] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 4
I20260709 13:00:08.505388  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.506417  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.509486  8817 pending_rounds.cc:107] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:08.513451  8817 raft_consensus.cc:2889] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:08.513577  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 117867us
I20260709 13:00:08.514081  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.514263 10431 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.514595  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:08.515026 10425 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.515234  8824 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:08.515460  8823 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:08.516876 10249 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.517261 10349 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.503577 10378 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.517963 10326 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.496363 10180 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.520246 10180 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.522480  8820 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:08.523872  8821 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:08.524958 10289 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.525614 10423 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.526906 10423 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:08.529165 10200 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.542016 10235 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.541538 10249 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.558264 10445 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.562119 10445 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.565887 10445 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.567552 10432 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.571357 10391 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.574294 10437 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:08.574200 10391 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:08.574932 10437 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:08.595198  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 1.21281e+06us
I20260709 13:00:08.570971 10286 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.600859 10443 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:08.601665 10443 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.602165  8864 leader_election.cc:400] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:08.602582  8864 leader_election.cc:403] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:08.605544 10435 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:00:08.607690  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 248) took 1475 ms. Trace:
I20260709 13:00:08.608135  8898 rpcz_store.cc:276] 0709 13:00:07.131954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.359867 (+227913us) service_pool.cc:224] Handling call
0709 13:00:07.371546 (+ 11679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:07.372176 (+   630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:07.372212 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:07.372750 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:00:07.648986 (+276236us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:07.649043 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:07.649139 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:08.607409 (+958270us) raft_consensus.cc:1685] finished
0709 13:00:08.607446 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.607616 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9984,"raft.queue_time_us":4447,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":25442,"spinlock_wait_cycles":3840,"thread_start_us":77762,"threads_started":1,"wal-append.queue_time_us":1134485}
I20260709 13:00:08.614464 10437 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.614753 10216 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
W20260709 13:00:08.615240  8863 connection.cc:256] RPC call timeout handler was delayed by -1.4118801789999793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
I20260709 13:00:08.615523 10391 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.617125 10364 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.618746 10260 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.619378 10260 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.622905 10447 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.623311  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.623649 10447 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.626024 10364 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.626645 10364 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.627137 10428 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.632263 10260 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.632833 10443 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.632957 10197 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.632334 10423 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:08.646988  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:08.647945  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:08.648869  8743 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:08.648722  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:08.651093 10447 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:08.667335  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 200089us
W20260709 13:00:08.669066  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 194386us
I20260709 13:00:08.671814  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.655074 10216 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.662309 10256 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.692663  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:08.696240  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77977.5us
I20260709 13:00:08.671088 10423 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:08.704463  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 189888us
W20260709 13:00:08.709828  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46731.7us
I20260709 13:00:08.681154 10364 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.711819  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:08.715845  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.712023  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:08.711131 10295 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.724917 10314 tablet_copy_client.cc:538] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:08.725039 10446 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:08.725716 10446 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.720734 10216 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.713183  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:08.729162 10249 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.729882  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:08.732424  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:08.733557 10182 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:08.721598 10256 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.733989 10182 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:08.730001 10446 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.718466  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 384) took 1083 ms. Trace:
I20260709 13:00:08.745846  8743 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:08.723894  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.727876  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.751739  8890 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:08.728217 10295 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.752981 10249 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.753701 10249 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.756264 10249 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:08.717398 10443 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:00:08.742468 10182 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.761444  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 227) took 1222 ms. Trace:
I20260709 13:00:08.762038  8817 rpcz_store.cc:276] 0709 13:00:07.539365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.539512 (+   147us) service_pool.cc:224] Handling call
0709 13:00:07.540000 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.514038 (+974038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.761085 (+247047us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400075654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:08.761360 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100587904}
I20260709 13:00:08.742039 10437 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:08.767438 10437 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:08.750557 10200 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.770488 10200 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:08.777048 10260 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.777855 10260 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.779188  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:08.780537 10260 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.768678 10437 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.791318 10202 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:08.793246  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45743.9us
I20260709 13:00:08.794381 10364 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:08.795145  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89241.3us
I20260709 13:00:08.797134 10364 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.797991  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:08.801522  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:08.802055 10216 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:08.791587 10256 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.802148 10364 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:08.802500  9441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.022s	sys 0.032s
W20260709 13:00:08.922181  9441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.031s	sys 0.072s
I20260709 13:00:08.922370  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:08.902318 10271 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.923864 10314 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 13:00:08.926242  8744 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:08.815414  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 104453us
W20260709 13:00:08.874861  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44068.4us
W20260709 13:00:08.901197  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44912us
I20260709 13:00:08.906162  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:08.870326  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.833245  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:08.831810  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:08.940973  8740 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:08.927704  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:08.773214 10200 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:08.834679  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.945688  8897 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:08.907498  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.794898 10391 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:08.941303  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 129592us
I20260709 13:00:08.888209  8898 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:08.795106 10249 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:08.828104 10446 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:08.957934  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 103416us
I20260709 13:00:08.799654 10260 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:08.963613  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.964310  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:08.964689  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.965065  8898 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.958988  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:08.831444  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.827512  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:08.969197  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:08.965268  9447 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:08.885967 10335 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:08.928933 10197 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:00:08.922749  8646 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:09.124683  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:09.154932  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:09.018448  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 231) took 1367 ms. Trace:
W20260709 13:00:09.165722  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:09.037743  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:09.140875  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 180605us
I20260709 13:00:09.122015  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.166846  8898 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.034684  8744 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:09.126109  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.034425  8742 raft_consensus.cc:2368] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3.
I20260709 13:00:09.169046  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.190742  8898 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.070761  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:08.995404  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:09.173352  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.165978  8822 rpcz_store.cc:276] 0709 13:00:07.591033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.609770 (+ 18737us) service_pool.cc:224] Handling call
0709 13:00:07.707794 (+ 98024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.544577 (+836783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.544627 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:08.544689 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:08.544736 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.904728 (+359992us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b540001f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:08.904792 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.919848 (+ 15056us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909138688}
I20260709 13:00:09.197572  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.198063  8898 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:09.198638  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
W20260709 13:00:09.199400  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128700us
I20260709 13:00:09.070715 10378 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:09.201534 10256 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:09.031749  8816 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:08.975059 10260 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:09.037499  8889 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:09.036114  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.218804  9443 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:09.219781  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:08.996490  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.147169  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 230) took 1604 ms. Trace:
I20260709 13:00:08.748013  9442 rpcz_store.cc:276] 0709 13:00:07.632595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.635884 (+  3289us) service_pool.cc:224] Handling call
0709 13:00:08.372620 (+736736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.376976 (+  4356us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:08.492520 (+115544us) log.cc:844] Serialized 14 byte log entry
0709 13:00:08.695938 (+203418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:08.698176 (+  2238us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:08.704117 (+  5941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.711088 (+  6971us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.714997 (+  3909us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8068,"prepare.queue_time_us":146866,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":2346,"raft.queue_time_us":6101,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5621,"thread_start_us":257784,"threads_started":2,"wal-append.queue_time_us":277946}
I20260709 13:00:09.145927  8745 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:08.968257 10391 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:09.204351 10271 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.124521 10182 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:09.205426  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 236678us
I20260709 13:00:08.969099 10249 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:09.150812  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.206130  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.206934  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.080068 10295 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:09.036550  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.212497  8823 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:09.212824  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:09.214694  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.215678 10256 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:09.218153  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.266180 10446 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:09.266497  8889 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:09.121371  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.224223  8740 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:09.226158  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.264076  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.229609  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.267915  8895 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.268062  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.268579 10254 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:09.268704  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.269340  8895 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:09.223460  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.213416 10454 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.210521 10386 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.204454  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.273226  8863 leader_election.cc:400] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:09.273783  8863 leader_election.cc:403] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:09.217236 10260 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:09.241454  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.241552  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:09.274454  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61740.9us
I20260709 13:00:09.241777 10335 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:09.241854 10182 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:09.221944 10437 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:00:09.231626 10271 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:09.265316  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50699.6us
I20260709 13:00:09.278874  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.271552  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:09.237607 10216 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:00:09.273634  8744 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:09.234262  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.279733  8740 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:09.276612  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.276897  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.274577  8747 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.228169  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.223810  8815 rpcz_store.cc:276] 0709 13:00:07.543134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.545886 (+  2752us) service_pool.cc:224] Handling call
0709 13:00:07.546347 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.397595 (+851248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.405894 (+  8299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:08.411127 (+  5233us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:08.411179 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.146397 (+735218us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:09.146474 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.147100 (+   626us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1937489280}
W20260709 13:00:09.278321 10423 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.004s
I20260709 13:00:09.285548  8742 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.281592 10271 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.293905  8646 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 2, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:09.071741  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:09.299127  8894 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:09.290583  8647 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 282) took 5307 ms. Trace:
I20260709 13:00:09.278714  8896 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:09.281438  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.266225  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:09.305594  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.277915 10454 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:09.293107  8891 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.273211  9445 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:09.275676 10254 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:09.312722  8895 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
W20260709 13:00:09.314477  8818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1093) took 2717 ms (client timeout 2182 ms). Trace:
I20260709 13:00:09.276612  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:09.318107  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:09.315078  8818 rpcz_store.cc:269] 0709 13:00:06.597556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.613550 (+ 15994us) service_pool.cc:224] Handling call
0709 13:00:09.191879 (+2578329us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:09.314388 (+122509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5245184}
I20260709 13:00:09.289523 10325 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.301680 10249 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:09.332130  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:09.326666  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:09.324097  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56791.1us
I20260709 13:00:09.312302  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.335433  8745 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:09.331115 10182 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:09.280164 10451 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:00:09.310318 10423 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.337639  8712 leader_election.cc:400] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:09.347924  8712 leader_election.cc:403] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:09.344882  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.322922 10378 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.307966  8746 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:09.329658 10437 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:09.287124 10256 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:09.299768  8647 rpcz_store.cc:276] 0709 13:00:03.982856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.983060 (+   204us) service_pool.cc:224] Handling call
0709 13:00:09.290365 (+5307305us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:05.468578 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:05.496248 (+ 27670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:05.496283 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:00:05.496303 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:05.496641 (+   338us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:05.496694 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:05.496720 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:05.496889 (+   169us) tablet.cc:701] Row locks acquired
0709 13:00:05.496903 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:00:05.497034 (+   131us) write_op.cc:270] Start()
0709 13:00:05.497132 (+    98us) write_op.cc:276] Timestamp: P: 1783602005496992 usec, L: 0
0709 13:00:05.497153 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:00:05.497813 (+   660us) log.cc:844] Serialized 536 byte log entry
0709 13:00:06.370856 (+873043us) op_driver.cc:464] REPLICATION: finished
0709 13:00:07.769232 (+1398376us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b18001d14c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:07.769775 (+   543us) write_op.cc:301] APPLY: starting
0709 13:00:07.769895 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:07.770080 (+   185us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:07.770130 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:07.770510 (+   380us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:07.770902 (+   392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:07.770990 (+    88us) write_op.cc:312] APPLY: finished
0709 13:00:07.942373 (+171383us) log.cc:844] Serialized 23 byte log entry
0709 13:00:08.358904 (+416531us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:08.372195 (+ 13291us) write_op.cc:454] Released schema lock
0709 13:00:08.372270 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:08.382346 (+ 10076us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:08.383970 (+  1624us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":448325,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":366931,"apply.run_cpu_time_us":14677,"apply.run_wall_time_us":607416,"num_ops":1,"prepare.queue_time_us":462838,"prepare.run_cpu_time_us":113966,"prepare.run_wall_time_us":203930,"raft.queue_time_us":459808,"raft.run_cpu_time_us":27340,"raft.run_wall_time_us":537808,"replication_time_us":793382,"spinlock_wait_cycles":19187072,"thread_start_us":1559678,"threads_started":5,"wal-append.queue_time_us":728171,"wal-append.run_cpu_time_us":224230,"wal-append.run_wall_time_us":1453152}]]}
I20260709 13:00:09.353457  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 386) took 1039 ms. Trace:
I20260709 13:00:09.339154 10349 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.365010  9446 rpcz_store.cc:276] 0709 13:00:08.307728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.462357 (+154629us) service_pool.cc:224] Handling call
0709 13:00:08.487033 (+ 24676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.561311 (+ 74278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.561342 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:08.562463 (+  1121us) log.cc:844] Serialized 26 byte log entry
0709 13:00:08.599286 (+ 36823us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:08.599323 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.035609 (+436286us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b1800408ac0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:09.346994 (+311385us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b54005b09d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:09.347072 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:09.347132 (+    60us) raft_consensus.cc:1685] finished
0709 13:00:09.347153 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.347338 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41534,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3850,"spinlock_wait_cycles":13282560,"thread_start_us":66867,"threads_started":2,"wal-append.queue_time_us":133021}
I20260709 13:00:09.342135  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.348606 10425 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.365095  8863 leader_election.cc:400] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:09.371377  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.371892  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.347683 10423 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.367977  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.329558 10325 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.354470  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.373571  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.372478  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.350071 10391 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:09.349366 10378 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.357167 10249 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:09.356465 10437 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:09.381027  9449 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:09.386143  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.369985 10349 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.378178 10335 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:09.380115 10182 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.381062 10189 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:09.381361 10423 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.373245 10425 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:09.387243 10378 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.401705  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.382185 10325 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.383530 10401 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.389668 10437 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.419996  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:09.401087 10473 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:00:09.408556 10254 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:09.401616 10424 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:09.402520 10349 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:09.427922  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 73562.4us
I20260709 13:00:09.432358  8744 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.431173  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.405167 10182 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.400719 10335 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:09.433228  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:09.414763 10425 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.437664  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:09.437868  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.438295  8890 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:09.438915  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.439219  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:09.439767  8890 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.440033  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.441602  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 462) took 1509 ms. Trace:
I20260709 13:00:09.436196  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.404029 10189 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:09.409391 10448 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:09.435456  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70552.8us
I20260709 13:00:09.442075  9441 rpcz_store.cc:276] 0709 13:00:07.932909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.952301 (+ 19392us) service_pool.cc:224] Handling call
0709 13:00:07.952966 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.922684 (+969718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.922722 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.195618 (+272896us) log.cc:844] Serialized 26 byte log entry
0709 13:00:09.247408 (+ 51790us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:09.247448 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.247524 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:09.441388 (+193864us) raft_consensus.cc:1685] finished
0709 13:00:09.441415 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.441548 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":379338,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":154,"thread_start_us":944462,"threads_started":2,"wal-append.queue_time_us":51253}
I20260709 13:00:09.443238 10335 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.445695 10249 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:09.445937  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:09.445029 10343 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.420878 10401 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.434995 10424 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:09.447750 10471 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:00:09.453367  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.444343 10182 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.408598  8745 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:00:09.448362  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:09.460292 10472 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:00:09.461069 10193 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.466275 10454 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:10.087983  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.014s	sys 0.035s
W20260709 13:00:10.112262  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.016s	sys 0.036s
I20260709 13:00:09.547079  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.114691  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:09.834363  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:09.495733  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48299.5us
I20260709 13:00:10.117731  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:09.448135 10286 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:09.828771  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.119871  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.128463  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:09.667299  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.130800  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1766) took 1766 ms. Trace:
I20260709 13:00:10.133667  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.134253  8898 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.134281 10489 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.134912 10489 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.135788  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:10.136752 10440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.002s	sys 0.009s
I20260709 13:00:10.136977  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:10.138100 10440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.002s	sys 0.009s
I20260709 13:00:10.137740 10489 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:10.138470  8822 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:10.138599 10489 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.139336 10489 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.140208  8898 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:10.131186  8823 rpcz_store.cc:276] 0709 13:00:08.364728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.516393 (+151665us) service_pool.cc:224] Handling call
0709 13:00:08.517014 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.264438 (+747424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.130598 (+866160us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54003e35d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:10.130749 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593838464}
I20260709 13:00:10.129866  8896 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:10.117090  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 395) took 1165 ms. Trace:
I20260709 13:00:10.144733 10482 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.454006 10401 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.144326  9443 rpcz_store.cc:276] 0709 13:00:08.951658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.220697 (+269039us) service_pool.cc:224] Handling call
0709 13:00:09.221273 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:09.233144 (+ 11871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.233177 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.234438 (+  1261us) log.cc:844] Serialized 564 byte log entry
0709 13:00:09.274148 (+ 39710us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.276929 (+  2781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.277010 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:10.111652 (+834642us) raft_consensus.cc:1685] finished
0709 13:00:10.111676 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.112586 (+   910us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":31096,"threads_started":1,"wal-append.queue_time_us":94473}
W20260709 13:00:10.150758 10425 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.000s
I20260709 13:00:10.151273  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:10.151135 10425 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.152281  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.153334  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.153869  8742 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:10.153966  8890 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.004s
I20260709 13:00:10.154349  8890 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:09.511468 10366 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.154646  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:10.154870 10366 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:10.155234  8742 raft_consensus.cc:2376] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:09.515345  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.156257  9442 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:10.155300 10366 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:10.147801 10349 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.000s
I20260709 13:00:10.156929 10349 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.145340 10482 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:10.157907 10482 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.159256 10349 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.147801  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.160681  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1424) took 1007 ms. Trace:
I20260709 13:00:10.161448  8818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44b3244bf12f42aba79421575a96c621"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:10.161860  8893 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.162225  8818 raft_consensus.cc:606] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:10.162348  8891 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.836s	user 0.005s	sys 0.005s
I20260709 13:00:10.162496  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1377) took 1327 ms. Trace:
I20260709 13:00:10.162627  8818 raft_consensus.cc:614] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:10.160988  8747 rpcz_store.cc:276] 0709 13:00:09.153407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.129918 (+976511us) service_pool.cc:224] Handling call
0709 13:00:10.160626 (+ 30708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.457983 10424 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.452169 10343 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:09.681914 10474 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
W20260709 13:00:09.704988 10478 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 13:00:10.156467 10335 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.662s	user 0.006s	sys 0.000s
I20260709 13:00:10.180102  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:09.555747 10476 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:00:09.569582 10456 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:00:09.456264 10480 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.156437  8742 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:09.670496 10216 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.181195  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.184098  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.170007  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 251) took 3031 ms. Trace:
I20260709 13:00:10.184916  8822 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:09.454205 10448 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:09.669906  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:10.176849  9441 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:10.153092 10425 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:09.497021  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.191150  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1395) took 1126 ms. Trace:
I20260709 13:00:09.498039 10391 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:10.194950  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:10.195432  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 465) took 1544 ms (client timeout 1597 ms). Trace:
I20260709 13:00:10.195245 10391 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.195824  8823 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.196262 10216 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.184092 10480 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:10.197667 10480 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.197808 10391 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.198297 10216 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:10.199465  8714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.002s	sys 0.005s
I20260709 13:00:09.681977 10446 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:09.682281 10483 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.831911  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.142328 10447 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:10.181864  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 200245us
I20260709 13:00:09.545585 10271 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:09.736565 10378 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:10.171792  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51999.1us
I20260709 13:00:10.177006 10343 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:10.175879  8715 connection.cc:256] RPC call timeout handler was delayed by -1.024945817999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:00:09.919685 10325 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.193727  8746 raft_consensus.cc:2376] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:10.198618 10391 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:09.978498 10249 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:10.167645  8893 rpcz_store.cc:276] 0709 13:00:08.834606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.867473 (+ 32867us) service_pool.cc:224] Handling call
0709 13:00:10.161432 (+1293959us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:10.161631 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.162261 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.162445 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9596160}
I20260709 13:00:10.207471  8895 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:10.200129  8894 rpcz_store.cc:276] 0709 13:00:09.064989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.307110 (+242121us) service_pool.cc:224] Handling call
0709 13:00:09.307468 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:09.317564 (+ 10096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.317584 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.322598 (+  5014us) log.cc:844] Serialized 26 byte log entry
0709 13:00:09.370182 (+ 47584us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:09.400026 (+ 29844us) log.cc:844] Serialized 14 byte log entry
0709 13:00:09.400398 (+   372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.400495 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:10.191015 (+790520us) raft_consensus.cc:1685] finished
0709 13:00:10.191027 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.191109 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9720,"prepare.queue_time_us":33607,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":10002,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":26694016,"thread_start_us":47248,"threads_started":1,"wal-append.queue_time_us":99667}
W20260709 13:00:10.209055 10437 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.788s	user 0.005s	sys 0.002s
I20260709 13:00:10.208926  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:10.183530 10335 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.200307 10401 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.209828 10193 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:09.504577 10189 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.184757  8892 rpcz_store.cc:276] 0709 13:00:07.132953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.374364 (+241411us) service_pool.cc:224] Handling call
0709 13:00:07.374790 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.465277 (+1090487us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:08.673346 (+208069us) log.cc:844] Serialized 14 byte log entry
0709 13:00:09.192391 (+519045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.192445 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:09.192500 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.164529 (+972029us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003be054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:10.164590 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.164734 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1560,"prepare.queue_time_us":247999,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":5132,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5958,"spinlock_wait_cycles":2486945152,"thread_start_us":649113,"threads_started":2,"wal-append.queue_time_us":545788}
I20260709 13:00:10.192315  8891 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:10.226217  8715 leader_election.cc:400] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:10.229686 10489 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:10.228313  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.257747  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.212786 10483 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:10.216503 10447 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:10.220017 10454 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.274195  8741 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:10.274484  8819 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:10.275224  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 54354.8us
W20260709 13:00:10.288820  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62809us
I20260709 13:00:10.265933 10193 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.215679 10366 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.294831  8894 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:10.217778 10446 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.277029 10447 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.299485  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:10.267122 10486 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:10.299444  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.276933 10483 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.289754 10189 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.314378  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:10.195850  9444 rpcz_store.cc:269] 0709 13:00:08.651114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.664983 (+ 13869us) service_pool.cc:224] Handling call
0709 13:00:09.495009 (+830026us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aae9c 00007f90f33aaa5b 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:10.195358 (+700349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8054016}
I20260709 13:00:10.338165  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.338896  8894 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:10.287712 10260 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3.
I20260709 13:00:10.312461 10249 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.264364 10391 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.318771  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.264746 10437 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.320894 10378 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.321616 10200 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.321877 10366 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.323382 10325 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.324713  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.295995  8817 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.326027 10271 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:10.326974 10423 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.906s	user 0.007s	sys 0.003s
I20260709 13:00:10.328918 10446 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.330343 10401 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.354207  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 393) took 1407 ms. Trace:
I20260709 13:00:10.354300 10423 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.304371  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1418) took 1154 ms. Trace:
I20260709 13:00:10.331573  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1423) took 1182 ms. Trace:
I20260709 13:00:10.331610  8821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:10.334085 10448 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.335894 10335 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.336172  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.375113  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.337150  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.375643  8816 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.376415  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.376950  8892 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:10.369078  8821 raft_consensus.cc:606] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:10.337832  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46095.4us
I20260709 13:00:10.345126 10492 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:10.288141 10454 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.297080  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1390) took 1258 ms. Trace:
I20260709 13:00:10.347848  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.330602 10193 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.324628 10482 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.354928  9440 rpcz_store.cc:276] 0709 13:00:08.946513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.951224 (+  4711us) service_pool.cc:224] Handling call
0709 13:00:10.303840 (+1352616us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:10.354114 (+ 50274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8847232}
I20260709 13:00:10.358024 10489 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:10.358644 10423 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.362185  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.362389 10391 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:10.363211 10497 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:10.364316  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70418.3us
I20260709 13:00:10.372494  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:10.396016  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:10.375680  8893 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:10.381814  8749 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.384083  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.337931 10182 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.391021  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:10.391747  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 1.2012e+06us
I20260709 13:00:10.363940  8748 rpcz_store.cc:276] 0709 13:00:09.147565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.147835 (+   270us) service_pool.cc:224] Handling call
0709 13:00:10.208676 (+1060841us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:10.301840 (+ 93164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6367104}
I20260709 13:00:10.344309 10343 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.390580  8891 rpcz_store.cc:276] 0709 13:00:09.037001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.292643 (+255642us) service_pool.cc:224] Handling call
0709 13:00:09.292954 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.285163 (+992209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.295790 (+ 10627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.403304  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:10.401952  8741 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:10.405881  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.406414  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:10.409240  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.409991  8820 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:10.411134  8892 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:10.412480  8822 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:10.344527 10486 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.414321  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:10.415117  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1436) took 1122 ms. Trace:
I20260709 13:00:10.415146  8893 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:10.415598  8743 rpcz_store.cc:276] 0709 13:00:09.292883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.414043 (+1121160us) service_pool.cc:224] Handling call
0709 13:00:10.415060 (+  1017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.416611  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.417920  8646 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 195) took 6226 ms. Trace:
I20260709 13:00:10.418668 10486 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.399791 10489 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.418274  8646 rpcz_store.cc:276] 0709 13:00:04.191925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.192103 (+   178us) service_pool.cc:224] Handling call
0709 13:00:04.533559 (+341456us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:00:10.417862 (+5884303us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:09.343304 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:09.343873 (+   569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:09.343947 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:00:09.343975 (+    28us) tablet.cc:662] Decoding operations
0709 13:00:09.344455 (+   480us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:09.344561 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:09.344589 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:00:09.345024 (+   435us) tablet.cc:701] Row locks acquired
0709 13:00:09.345059 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:00:09.345205 (+   146us) write_op.cc:270] Start()
0709 13:00:09.345383 (+   178us) write_op.cc:276] Timestamp: P: 1783602009345150 usec, L: 0
0709 13:00:09.345411 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:00:09.346130 (+   719us) log.cc:844] Serialized 919 byte log entry
0709 13:00:10.134279 (+788149us) op_driver.cc:464] REPLICATION: finished
0709 13:00:10.303772 (+169493us) write_op.cc:301] APPLY: starting
0709 13:00:10.317429 (+ 13657us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:10.317564 (+   135us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:10.317739 (+   175us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:10.320940 (+  3201us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:10.321108 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:00:10.321153 (+    45us) write_op.cc:312] APPLY: finished
0709 13:00:10.321395 (+   242us) log.cc:844] Serialized 29 byte log entry
0709 13:00:10.321567 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.321663 (+    96us) write_op.cc:454] Released schema lock
0709 13:00:10.321706 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:10.322064 (+   358us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:10.322104 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":919233,"reported_tablets":5,"spinlock_wait_cycles":578321792,"child_traces":[["op",{"apply.queue_time_us":155707,"apply.run_cpu_time_us":3461,"apply.run_wall_time_us":22690,"num_ops":2,"prepare.queue_time_us":28849,"prepare.run_cpu_time_us":3122,"prepare.run_wall_time_us":3123,"replication_time_us":788582,"thread_start_us":168994,"threads_started":2}]]}
I20260709 13:00:10.421867 10489 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.424017 10454 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.424520  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.425369 10401 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.425226 10424 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.426090 10401 raft_consensus.cc:993] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: : Instructing follower b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 13:00:10.426327  8824 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.426569 10401 raft_consensus.cc:1081] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 13:00:10.431149  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.431286 10447 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.432909 10437 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.433202  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.421895  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1396) took 1356 ms. Trace:
I20260709 13:00:10.448889  8898 rpcz_store.cc:276] 0709 13:00:09.065387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.140759 (+1075372us) service_pool.cc:224] Handling call
0709 13:00:10.142935 (+  2176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.143088 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.143103 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.143128 (+    25us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:10.143145 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.143177 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.421747 (+278570us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b54003723d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:10.421842 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4418304}
I20260709 13:00:10.457047  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.406500 10423 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:10.428618 10424 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.408675 10249 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.470311  8744 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:10.467338  8864 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.593s (SENT)
I20260709 13:00:10.471865  8749 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:10.433261 10447 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.473479  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1422) took 1324 ms. Trace:
I20260709 13:00:10.473847  8749 rpcz_store.cc:276] 0709 13:00:09.148807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.381365 (+1232558us) service_pool.cc:224] Handling call
0709 13:00:10.381686 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.473214 (+ 91528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.473424 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.435314 10343 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:10.478312  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 75105.8us
I20260709 13:00:10.436195 10182 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.437476  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 401) took 1076 ms. Trace:
I20260709 13:00:10.479879  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.470966 10325 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:10.473774  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89778.9us
I20260709 13:00:10.483973  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.486351  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:10.486843  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1420) took 1338 ms. Trace:
I20260709 13:00:10.483995  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.481498 10492 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:10.482002  8748 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:10.479211 10423 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:10.487236  8740 rpcz_store.cc:276] 0709 13:00:09.148217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.380323 (+1232106us) service_pool.cc:224] Handling call
0709 13:00:10.380609 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.380790 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:10.412010 (+ 31220us) log.cc:844] Serialized 14 byte log entry
0709 13:00:10.486545 (+ 74535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.486594 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:10.486632 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.486690 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.486794 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9397,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"thread_start_us":44050,"threads_started":2}
I20260709 13:00:10.368588  8745 rpcz_store.cc:276] 0709 13:00:09.149129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.336135 (+187006us) service_pool.cc:224] Handling call
0709 13:00:09.336474 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.352643 (+ 16169us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:10.208174 (+855531us) log.cc:844] Serialized 14 byte log entry
0709 13:00:10.288890 (+ 80716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.290935 (+  2045us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.293519 (+  2584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.299071 (+  5552us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.318303 (+ 19232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15981,"raft.queue_time_us":19094,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":11333,"thread_start_us":55590,"threads_started":1,"wal-append.queue_time_us":137426}
I20260709 13:00:10.435887 10480 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.400483 10497 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.489984  8822 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:10.504109  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101882us
I20260709 13:00:10.495956  8746 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:10.501633  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.486114 10325 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:10.507179  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.508860  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.509085  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.509423  8892 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.509613  8898 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.491427  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.495520 10480 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.508569 10325 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.491912 10492 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:10.553229  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48338.1us
I20260709 13:00:10.552778  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.553104  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.553773 10378 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:10.554441 10497 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:10.555564 10482 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:10.556377 10193 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.559588  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.504876 10486 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.560693  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:10.590516  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.558398  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.558684  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.595237  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.595736  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:10.588662  8889 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.596570  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.597220  9444 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
W20260709 13:00:10.588454 10502 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.197s	user 0.004s	sys 0.006s
I20260709 13:00:10.599047 10425 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:10.599539 10425 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.560612  8898 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:10.571177 10314 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:00:10.572644  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:10.490497 10484 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:00:10.577818  8897 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.505797 10454 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:00:10.557261 10483 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.483850  9441 rpcz_store.cc:276] 0709 13:00:09.361257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.494368 (+133111us) service_pool.cc:224] Handling call
0709 13:00:09.661602 (+167234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.195154 (+533552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.437213 (+242059us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:10.437412 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082839936}
W20260709 13:00:10.593683  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.593851 10487 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.594695  8821 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:10.661985  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101373us
I20260709 13:00:10.664275  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.721254  8896 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.662348  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:10.722257  8822 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 4.
I20260709 13:00:10.660471  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.656299  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.645996  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.723241  8898 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:10.727150  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 154547us
I20260709 13:00:10.728572  8789 leader_election.cc:400] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3.
W20260709 13:00:10.726718  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45174.6us
I20260709 13:00:10.734405  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:10.735427  9446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 402) took 1371 ms (client timeout 1516 ms). Trace:
W20260709 13:00:10.735852  9446 rpcz_store.cc:269] 0709 13:00:09.363767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.378138 (+ 14371us) service_pool.cc:224] Handling call
0709 13:00:10.660295 (+1282157us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:10.735370 (+ 75075us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3134208}
W20260709 13:00:10.654441 10506 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:00:10.745085  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.749559  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:10.743089  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:10.751319  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 191525us
I20260709 13:00:10.744148  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:10.752889  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.754115  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.754771  8822 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:10.754644  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.755642  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.756343  8749 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:10.757699 10437 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:10.726644 10497 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.728175 10487 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:10.758291 10437 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:10.752238 10480 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:00:10.759274  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.759574 10480 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:10.759891  8816 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:10.759981 10480 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:10.762338  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.759605 10437 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.763556  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.764633  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.765429 10510 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.688371  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 398) took 1304 ms. Trace:
I20260709 13:00:10.761082 10480 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.767031  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.768215  8897 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:10.768816 10480 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.769129  8822 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:10.601047  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.767791  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.659140  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.770663  8823 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:10.758704 10487 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:10.790434  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62024.9us
I20260709 13:00:10.790750  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.792325  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.777213  8892 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:10.793830  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.793932  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.777840  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.632238 10425 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:10.633479  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.770682  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.799906  8893 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.800088  8821 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:10.779062  8818 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:10.779170 10386 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.760965  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.783931  8889 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:10.801934 10386 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:10.802160  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.802455 10386 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.802829  8889 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.784509  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:10.786870 10483 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.744052 10378 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 13:00:10.788442  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.788133 10497 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.813126 10386 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:10.814386 10497 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:10.816211 10386 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:10.811612  8889 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:10.823647 10286 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.838634 10286 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:10.839075  9414 connection.cc:256] RPC call timeout handler was delayed by -1.0065586619999749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 13:00:10.839165 10286 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.818840  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:10.819240  8823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:10.816988  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.841816  8892 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.842533  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.825702  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:10.822788  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.844022  8822 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:10.831662  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:10.834450  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 90373.7us
I20260709 13:00:10.600543 10423 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:10.853529  8815 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:10.854434  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.747521 10193 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:00:10.856053  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.856652  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.743820 10482 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:00:10.817636 10486 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:00:10.824677 10480 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 13:00:10.841894  8823 raft_consensus.cc:606] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:10.910369  8892 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:10.900321  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:10.885874  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60283us
W20260709 13:00:10.912235  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61583.4us
I20260709 13:00:10.857044  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:10.970487  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 127033us
I20260709 13:00:10.982164  8742 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:10.984580  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:10.991029  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 134333us
I20260709 13:00:10.992473  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:10.993256  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.953269 10314 tablet_bootstrap.cc:492] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 13:00:10.994510  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.996575  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.997643 10510 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.997854 10314 ts_tablet_manager.cc:1403] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 2.112s	user 0.042s	sys 0.035s
I20260709 13:00:10.998052 10189 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:10.912071 10508 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.911929 10378 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:10.994603  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:10.999258  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:10.921087 10256 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:10.883441  8643 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:10.883258  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.000974  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:11.004956  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:10.842993 10483 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.169s
I20260709 13:00:10.991214  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.034288  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:10.990460 10489 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.051380  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.052181  8742 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 1.
I20260709 13:00:11.008652  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.037830  9449 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.996102 10182 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:11.054692  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73947.5us
I20260709 13:00:11.055756  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.056291  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:11.039561  9446 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:11.057469  9446 pending_rounds.cc:85] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 4
I20260709 13:00:11.057798  9446 pending_rounds.cc:107] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:11.058099  9446 raft_consensus.cc:2889] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:11.058815  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.038439  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.059792  8742 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:11.059598  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:11.040251  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:11.038754  8643 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:11.049160  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:11.005599 10508 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:11.063458  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73406.2us
I20260709 13:00:11.006844 10454 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:11.041713  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:11.065999  9448 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:11.039968  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45572.2us
W20260709 13:00:11.064191  9414 leader_election.cc:336] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.701s (SENT)
I20260709 13:00:11.008082 10256 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:11.035419 10189 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:11.074985  9414 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.749s (SENT)
I20260709 13:00:11.126259  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.076229  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.126464  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:11.126343  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.039284 10378 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:11.077878  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.041631 10341 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.047924 10254 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.073375  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:11.153676  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80761.6us
I20260709 13:00:11.128103  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.132504  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.066687 10508 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:11.133953  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59002.9us
I20260709 13:00:11.124331 10483 consensus_queue.cc:1243] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:11.178824  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:00:11.179684  8898 raft_consensus.cc:606] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:00:11.180086  8898 raft_consensus.cc:614] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:11.123931  8743 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.141644 10256 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.075587 10437 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.144505  9449 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:11.145162  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.145392 10511 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.552s	user 0.004s	sys 0.000s
I20260709 13:00:11.146014 10378 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.065781 10454 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:11.063699 10489 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:11.072803 10182 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:11.125156  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 48129.1us
I20260709 13:00:11.071394  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:11.155220  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.155475  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.065671 10424 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.168982  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:11.158883  8741 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.065532 10197 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:11.178766  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119244us
I20260709 13:00:11.185968  8742 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.186348  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:11.186930  9414 leader_election.cc:336] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.917s (SENT)
I20260709 13:00:11.162382 10254 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:11.180155  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.189085 10437 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:11.190414  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:10.767267  9445 rpcz_store.cc:276] 0709 13:00:09.358621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.363264 (+  4643us) service_pool.cc:224] Handling call
0709 13:00:09.393069 (+ 29805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.421053 (+ 27984us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:09.454568 (+ 33515us) log.cc:844] Serialized 14 byte log entry
0709 13:00:10.551129 (+1096561us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180037cea0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:00:10.567816 (+ 16687us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.578693 (+ 10877us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:10.583849 (+  5156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.593299 (+  9450us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.663376 (+ 70077us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47956,"prepare.queue_time_us":16583,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1100337,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":28139776}
I20260709 13:00:11.519018  8822 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:11.518803 10254 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.520025  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.521461  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.521883  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.309172  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.522547  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.524399  9449 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.526309  8892 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.510093  9448 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:11.157570 10521 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:11.482575  8864 leader_election.cc:336] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 2.182s (SENT)
I20260709 13:00:11.543541  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:11.544539  8823 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
W20260709 13:00:11.265758  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.520982 10314 raft_consensus.cc:359] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.361907  8898 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.562049  8898 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.563472  8898 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.310765  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:11.531328  8896 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.000s
W20260709 13:00:11.546466  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.547108  8747 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:11.319875  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 187) took 1018 ms. Trace:
I20260709 13:00:11.194830 10189 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.558291 10279 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.559748  8820 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:11.431221 10286 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
W20260709 13:00:11.583920  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101388us
I20260709 13:00:11.565876  8749 rpcz_store.cc:276] 0709 13:00:10.301232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.483750 (+182518us) service_pool.cc:224] Handling call
0709 13:00:11.319563 (+835813us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54000b5754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:11.319788 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620537600}
I20260709 13:00:11.568634  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1431) took 2314 ms. Trace:
I20260709 13:00:11.569659  8892 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.558930  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.590982  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.591197  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.592034  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:11.592253  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.320201  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:11.573907  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 386948us
I20260709 13:00:11.547931 10424 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:11.363942  8893 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.563s	user 0.004s	sys 0.004s
I20260709 13:00:11.613822  8893 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:11.570238  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.614535  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1409) took 1104 ms. Trace:
I20260709 13:00:11.615087  8822 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:11.615000  8893 rpcz_store.cc:276] 0709 13:00:10.510030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.551687 (+ 41657us) service_pool.cc:224] Handling call
0709 13:00:11.614439 (+1062752us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:11.426079 10493 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:00:11.582016  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.183795  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1401) took 1788 ms. Trace:
I20260709 13:00:11.308640  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.589660  8744 rpcz_store.cc:276] 0709 13:00:09.234753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.443357 (+208604us) service_pool.cc:224] Handling call
0709 13:00:09.452524 (+  9167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.475036 (+1022512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.548974 (+1073938us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400466ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:11.549199 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2339939712}
I20260709 13:00:11.608374  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:11.359839  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.426649  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.601902  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.542773 10335 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:11.608003 10325 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.037s	user 0.011s	sys 0.000s
I20260709 13:00:11.603014  9449 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:11.593091  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.604300  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44422us
I20260709 13:00:11.629037  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:11.559535 10521 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.631052 10521 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:11.632284  8788 connection.cc:256] RPC call timeout handler was delayed by -1.4708276869999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:00:11.632817  8788 connection.cc:256] RPC call timeout handler was delayed by -1.417479856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:00:11.574172  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:11.634486 10423 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:11.635052 10423 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:11.635515 10423 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:11.637509 10507 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:11.633176  8788 connection.cc:256] RPC call timeout handler was delayed by -1.3670798559999753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
W20260709 13:00:11.639995  8788 connection.cc:256] RPC call timeout handler was delayed by -1.3517798560000074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 13:00:11.608465  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.641738  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.640408  8788 connection.cc:256] RPC call timeout handler was delayed by -1.3126375639999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
I20260709 13:00:11.643113  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.642903  8788 connection.cc:256] RPC call timeout handler was delayed by -1.2311798559999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 13:00:11.598430  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:11.645238  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46895.1us
I20260709 13:00:11.646116  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:11.574234 10341 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.606105  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.648869  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.650228  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.601955  8863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.012s (SENT) [suppressed 64 similar messages]
I20260709 13:00:11.565011  8896 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:11.665908  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1412) took 1153 ms. Trace:
I20260709 13:00:11.666378  8896 rpcz_store.cc:276] 0709 13:00:10.512393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.663950 (+151557us) service_pool.cc:224] Handling call
0709 13:00:11.665821 (+1001871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.543183 10437 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.617674  8821 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:11.575042 10386 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.675264 10386 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.675834 10386 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.676252 10386 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.585497 10256 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.575421 10197 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:11.683219 10197 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.625007  8747 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:11.686214  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.686980  8747 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:11.619050  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:11.713641  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 111687us
I20260709 13:00:11.722117  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.724097  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.725574  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.726393  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.750180  8741 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:11.622956 10314 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 13:00:11.741921  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.620340  8890 rpcz_store.cc:276] 0709 13:00:09.392575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.155033 (+762458us) service_pool.cc:224] Handling call
0709 13:00:10.155354 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:10.207057 (+ 51703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.207086 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:10.207609 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:00:10.268631 (+ 61022us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.278052 (+  9421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.124599 (+846547us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b5400247a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:11.129263 (+  4664us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:11.131166 (+  1903us) raft_consensus.cc:1685] finished
0709 13:00:11.134209 (+  3043us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.180856 (+ 46647us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1280,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":103168768,"thread_start_us":55577,"threads_started":1,"wal-append.queue_time_us":155292,"wal-append.run_cpu_time_us":1921,"wal-append.run_wall_time_us":1074684}
I20260709 13:00:11.619709 10424 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.309324 10454 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:11.743381 10487 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.909s	user 0.010s	sys 0.000s
I20260709 13:00:11.727319  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.594496 10286 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.768106  8744 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.736557 10521 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.626343 10445 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:11.748724  8749 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
W20260709 13:00:11.747897  9440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.043s	sys 0.054s
I20260709 13:00:11.777048  8742 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.773098  8747 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:11.770521  8818 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:11.782842  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.778687  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.786370  8749 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:11.789183 10386 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:11.791688 10286 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:11.792543 10423 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.794111  8742 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:11.795295  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.795799  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:11.796801  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.797550  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.785130 10445 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.798527  8743 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:11.799525  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.745819 10335 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.732409 10437 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.805733  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.781041  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.812602  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:11.778066 10497 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.745652 10325 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:11.782478  9440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.044s	sys 0.055s
I20260709 13:00:11.814679  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:11.810257  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.817008  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.815542  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 93792.7us
W20260709 13:00:11.829692  9414 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.236s (SENT)
I20260709 13:00:11.830422  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.819351  8742 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:11.833410  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.835979  8818 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:11.837895  8822 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:11.845537  9414 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.253s (SENT)
I20260709 13:00:11.846136  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.847121  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.848367  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.786433  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.851341  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.851513  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.852981  8744 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.853866  8741 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:11.854801  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.855479  8741 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:11.856691  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.775223 10487 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.815538 10454 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.809939 10423 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:11.802377  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 269) took 1065 ms. Trace:
I20260709 13:00:11.861212  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.865213  8742 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:11.861559  8740 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:11.861451 10260 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.859934  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.867282 10445 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:11.868222 10386 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:11.868350  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.868824 10386 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:11.869578 10386 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.870286  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.871096 10386 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.872135  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:11.872231  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:11.873037 10386 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:11.873420 10386 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.873723 10386 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:11.785377 10182 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.880625  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.884358  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.885718  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:11.859433 10286 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:11.889212  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68290.8us
I20260709 13:00:11.878378 10335 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:11.891057  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 244992us
I20260709 13:00:11.893540  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:11.897572  8895 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:11.901469  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:11.902477 10335 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.901746  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:11.811031 10314 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:11.904518  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.888643  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:11.894877  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 255099us
I20260709 13:00:11.905567  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:11.907855  8744 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.873656  8748 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:11.903136  8815 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:11.903251 10486 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:11.951285  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49629.8us
I20260709 13:00:11.952594  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:11.928368 10341 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:11.948876 10486 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:11.951746 10507 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.955196 10489 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:11.962410 10448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.563s	user 0.011s	sys 0.018s
I20260709 13:00:11.967339  8743 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:11.967561  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.955866 10537 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:11.976963  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 95868.9us
W20260709 13:00:11.980121 10534 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:00:11.978657 10539 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:11.981278 10537 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:11.983603 10537 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:11.983991 10314 ts_tablet_manager.cc:1434] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.986s	user 0.009s	sys 0.010s
W20260709 13:00:11.984311 10314 ts_tablet_manager.cc:1467] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Tablet startup took 986ms
I20260709 13:00:11.953162 10254 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:00:11.980511  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:11.973909 10448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.576s	user 0.012s	sys 0.018s
I20260709 13:00:11.995350 10487 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:11.986603  8744 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:11.978576 10489 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.995543  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:12.009629  8898 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:11.991515 10546 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:11.996611  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.976188 10521 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:11.997838  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:11.981320 10539 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:11.989187  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:12.006939  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.026101  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:12.027483  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:11.975275  8743 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:11.986147 10481 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:12.005712 10437 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:12.002524 10533 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:00:11.866971  8891 rpcz_store.cc:276] 0709 13:00:10.721217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.511738 (+790521us) service_pool.cc:224] Handling call
0709 13:00:11.512161 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.512427 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:11.512894 (+   467us) log.cc:844] Serialized 14 byte log entry
0709 13:00:11.768406 (+255512us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.770318 (+  1912us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:11.773073 (+  2755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.778055 (+  4982us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.786809 (+  8754us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":270,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":49,"raft.queue_time_us":22104,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5359,"thread_start_us":232358,"threads_started":1,"wal-append.queue_time_us":270929}
I20260709 13:00:12.001942 10256 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:00:12.029247  8743 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.030685  8743 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.016490 10555 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.032362 10555 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.032727 10555 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.033063 10555 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.029588 10481 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.035487 10481 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:12.036837  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:12.004022 10424 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.001480  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 400) took 1002 ms. Trace:
I20260709 13:00:12.037468  9440 rpcz_store.cc:276] 0709 13:00:10.999542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.001722 (+  2180us) service_pool.cc:224] Handling call
0709 13:00:11.075876 (+ 74154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.137045 (+ 61169us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:11.851990 (+714945us) log.cc:844] Serialized 14 byte log entry
0709 13:00:11.964304 (+112314us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.964714 (+   410us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:11.975463 (+ 10749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.979089 (+  3626us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.998051 (+ 18962us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":624172,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3639,"raft.queue_time_us":2742,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":9008,"thread_start_us":731346,"threads_started":2}
I20260709 13:00:12.038755 10424 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.042689 10555 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.044579  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:12.045090 10555 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.045740  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.046119 10555 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.046504  8744 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:12.074380  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.074795  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:12.050310  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:12.054172  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.054708  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1780) took 2235 ms. Trace:
W20260709 13:00:12.056734 10480 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:00:12.060621  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 188) took 1756 ms. Trace:
I20260709 13:00:12.089704  8740 rpcz_store.cc:276] 0709 13:00:10.304095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.492879 (+188784us) service_pool.cc:224] Handling call
0709 13:00:10.493216 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.865792 (+1372576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:12.060389 (+194597us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54004632d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:12.060544 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3146262272}
I20260709 13:00:12.063274 10425 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:12.063827 10260 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.065279  8891 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:12.068606  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.071115  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.075802  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:12.085551  8789 connection.cc:256] RPC call timeout handler was delayed by -1.9489981520000583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:00:12.088975  8824 rpcz_store.cc:276] 0709 13:00:09.819257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.819377 (+   120us) service_pool.cc:224] Handling call
0709 13:00:09.826806 (+  7429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.426690 (+599884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:12.054406 (+1627716us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540012c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:12.054610 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378175872}
W20260709 13:00:12.096479  8789 connection.cc:256] RPC call timeout handler was delayed by -1.3821806910000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:00:12.096856  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2295857940000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23090000000000002s for a response
W20260709 13:00:12.097124  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1185019590000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 13:00:12.097421  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0339019590000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 13:00:12.097685  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0123036790000128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
I20260709 13:00:12.097961  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.099207  8822 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.095991  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:12.100549  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111430us
I20260709 13:00:12.100749  8740 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3. Re-sending same reply.
I20260709 13:00:12.104242  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.111713  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:12.115206 10545 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.157s	user 0.003s	sys 0.001s
I20260709 13:00:12.141682 10537 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.118777 10447 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.142830 10447 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:12.144249 10487 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.143970 10447 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.144757 10487 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.121433  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.146173  8745 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:12.145970 10487 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:12.146847  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1105) took 1312 ms (client timeout 1552 ms). Trace:
W20260709 13:00:12.147344  8745 rpcz_store.cc:269] 0709 13:00:10.834800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.855463 (+ 20663us) service_pool.cc:224] Handling call
0709 13:00:12.121281 (+1265818us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:12.146776 (+ 25495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5810048}
I20260709 13:00:12.134241  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:12.138733 10364 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:12.142310 10537 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:12.218014  8897 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.218288  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:12.218173 10537 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.218909  8889 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.147550 10487 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.219317  8897 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.220382  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:12.179121  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 186) took 1885 ms (client timeout 1749 ms). Trace:
I20260709 13:00:12.221626  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:12.178279  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:12.187294  8894 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.190680  8892 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.191883  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:12.125958  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.179385 10325 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.193949  8890 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.185999  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.213886 10529 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.232648 10529 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:12.233608 10529 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:12.232038  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1111) took 1673 ms (client timeout 1601 ms). Trace:
W20260709 13:00:12.203755  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69614.3us
I20260709 13:00:12.214466  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.149785 10447 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:12.221421  8746 rpcz_store.cc:269] 0709 13:00:10.293311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.324424 (+ 31113us) service_pool.cc:224] Handling call
0709 13:00:12.160376 (+1835952us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400465ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:12.179004 (+ 18628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326659456}
I20260709 13:00:12.182677 10341 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:12.074059 10555 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:12.002053 10486 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.267659  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:12.226962  8824 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.237625 10279 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:12.101760  8744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:51668
I20260709 13:00:12.270205  8744 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.270509  8744 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.271661  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.247973  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.102455  9443 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.284776  8747 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:12.257736  8820 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.257851  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.255965  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:12.264729  8823 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.289690  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:12.094023 10260 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:12.290439 10260 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.273198  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:12.258543 10552 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.333436 10564 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.333830 10564 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:12.334149  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.333834 10552 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.102155  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.335203 10571 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.335762  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 401) took 1335 ms. Trace:
I20260709 13:00:12.336114  9441 rpcz_store.cc:276] 0709 13:00:10.999911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.001238 (+  1327us) service_pool.cc:224] Handling call
0709 13:00:12.094244 (+1093006us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:12.335703 (+241459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21088000}
I20260709 13:00:12.336526 10552 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:12.239586 10480 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.337329 10480 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.339262 10480 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:12.272946  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 404) took 1272 ms. Trace:
I20260709 13:00:12.339848  9444 rpcz_store.cc:276] 0709 13:00:11.000920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.003816 (+  2896us) service_pool.cc:224] Handling call
0709 13:00:12.159641 (+1155825us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:12.272885 (+113244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20976000}
I20260709 13:00:12.273185 10554 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.273806  8822 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.341216 10554 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:12.341625 10554 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.343003  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.344344  8822 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.274245 10486 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.274911  8890 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:12.349789  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:12.350364  8819 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.274958  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.351766  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.351945 10564 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.276620 10568 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:12.353240 10564 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:11.984604 10314 ts_tablet_manager.cc:1469] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Trace:
0709 13:00:03.958155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.958273 (+   118us) service_pool.cc:224] Handling call
0709 13:00:04.142635 (+184362us) ts_tablet_manager.cc:934] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
0709 13:00:05.521208 (+1378573us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:06.023714 (+502506us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:06.025011 (+  1297us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:06.457706 (+432695us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:06.458049 (+   343us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:06.458397 (+   348us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:06.466619 (+  8222us) inbound_call.cc:177] Queueing success response
0709 13:00:08.832605 (+2365986us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:00:10.998281 (+2165676us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:00:11.150702 (+152421us) tablet_replica.cc:261] Starting instrumentation
0709 13:00:11.151332 (+   630us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":52402,"mutex_wait_us":16187,"raft.queue_time_us":886,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20134,"tablet-copy.queue_time_us":122136,"tablet-download-pool.queue_time_us":102422,"tablet-download-pool.run_cpu_time_us":2105,"tablet-download-pool.run_wall_time_us":1857870,"thread_start_us":274858,"threads_started":2}
I20260709 13:00:12.277252  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:12.354940  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77769.2us
W20260709 13:00:12.355430  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 83797.9us
I20260709 13:00:12.277551 10560 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.268576 10555 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:12.278462 10551 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:00:12.278846  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.279874  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1122) took 1154 ms. Trace:
I20260709 13:00:12.372395  8749 rpcz_store.cc:276] 0709 13:00:11.125644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.787079 (+661435us) service_pool.cc:224] Handling call
0709 13:00:11.813544 (+ 26465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:11.818314 (+  4770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.818357 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:11.858883 (+ 40526us) log.cc:844] Serialized 26 byte log entry
0709 13:00:11.959138 (+100255us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:11.959181 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.959255 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:12.279681 (+320426us) raft_consensus.cc:1685] finished
0709 13:00:12.279705 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:12.279823 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13243,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14215,"thread_start_us":67983,"threads_started":1,"wal-append.queue_time_us":351248}
I20260709 13:00:12.374732  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.249203 10485 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.282843  8889 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:12.282755 10386 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.256299  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:12.305013  8892 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:12.379233  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 121464us
W20260709 13:00:12.379753  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55675.2us
I20260709 13:00:12.380185  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.260591 10481 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.359936  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.310976  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.311487 10279 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.315384 10521 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:12.395537  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84642.6us
I20260709 13:00:12.290496 10569 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.320914 10539 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.322271  8820 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.298247 10325 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:00:12.327992  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80079.9us
I20260709 13:00:12.402496  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.095310  8743 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:12.329227  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1123) took 1171 ms. Trace:
I20260709 13:00:12.330099 10537 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.331207 10497 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.332429  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.332615 10558 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.332791 10556 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.358956  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:12.422852  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.326258  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:12.427991  8820 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:12.335238  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:12.236872  8817 rpcz_store.cc:269] 0709 13:00:10.558592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.566932 (+  8340us) service_pool.cc:224] Handling call
0709 13:00:12.159260 (+1592328us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:12.231925 (+ 72665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11863936}
W20260709 13:00:12.362815  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1865660769999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 13:00:12.364215  8819 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
W20260709 13:00:12.428997  8712 connection.cc:256] RPC call timeout handler was delayed by -1.156866076999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
I20260709 13:00:12.366685  8744 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:12.429418  8712 connection.cc:256] RPC call timeout handler was delayed by -1.0731660769999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:00:12.367630  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 304265us
W20260709 13:00:12.371201  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 245518us
W20260709 13:00:12.309063 10508 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.115s	user 0.006s	sys 0.006s
I20260709 13:00:12.355152 10501 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.379267 10364 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.377761 10560 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:12.387753 10481 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.390197 10489 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.392222  8816 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:12.426443 10556 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:12.393635  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.378500  8745 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:12.310506 10260 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:12.368876 10341 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.382901 10485 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.406865  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.384114 10454 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.408044  8741 rpcz_store.cc:276] 0709 13:00:11.157907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.890057 (+732150us) service_pool.cc:224] Handling call
0709 13:00:11.890364 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:11.990082 (+ 99718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.990116 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:11.990721 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:00:12.005670 (+ 14949us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:12.081488 (+ 75818us) log.cc:844] Serialized 14 byte log entry
0709 13:00:12.081710 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:12.081816 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:12.158688 (+ 76872us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b180040eca0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:12.323850 (+165162us) raft_consensus.cc:1685] finished
0709 13:00:12.328943 (+  5093us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:12.329151 (+   208us) inbound_call.cc:177] Queueing success response
0709 13:00:12.350400 (+ 21249us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b180040eca0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
Metrics: {"mutex_wait_us":8557,"prepare.queue_time_us":82588,"prepare.run_cpu_time_us":2012,"prepare.run_wall_time_us":20123,"raft.queue_time_us":75894,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":10027,"spinlock_wait_cycles":111205504,"thread_start_us":113758,"threads_started":2,"wal-append.queue_time_us":257159}
I20260709 13:00:12.423435  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:12.429551  8823 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:12.429808  8712 connection.cc:256] RPC call timeout handler was delayed by -1.0290660769999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260709 13:00:12.356300  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:12.456300  8712 connection.cc:256] RPC call timeout handler was delayed by -1.0191660769999658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
I20260709 13:00:12.396360 10256 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.430560  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:12.421667  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.396576 10521 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:12.433604  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:12.458264  8820 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:12.441784 10501 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:12.419368 10539 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.423821 10497 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.423988 10558 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:12.468058 10558 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.468190 10560 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.380607 10425 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:00:12.469426 10569 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.469947 10425 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:12.470098 10489 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.470350 10425 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:12.471274 10425 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.472411 10425 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.472710 10485 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:12.432596  9443 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:12.473348 10447 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.473562  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 406) took 1470 ms. Trace:
I20260709 13:00:12.473776 10447 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.474243 10447 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.473989  9443 rpcz_store.cc:276] 0709 13:00:11.003868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.006723 (+  2855us) service_pool.cc:224] Handling call
0709 13:00:12.070928 (+1064205us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:12.473499 (+402571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13659520}
I20260709 13:00:12.437786 10249 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:00:12.395373  8898 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:12.476037  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.476306 10521 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.437606 10538 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.477607  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.478013  8745 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.438995  8744 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.418582 10557 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:00:12.459051  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52280.4us
I20260709 13:00:12.479362  8745 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.479840  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:12.460886  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104670us
I20260709 13:00:12.461627 10510 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.452451 10341 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:12.461858  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68288.7us
I20260709 13:00:12.463413 10279 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:12.463763 10454 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.463460 10260 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:12.463649 10554 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:12.448738 10556 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.464982 10567 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:12.464459 10501 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:12.465145  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.437261 10508 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:12.476967  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53620us
I20260709 13:00:12.478519  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.479138 10564 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.490371  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:12.453024 10486 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.494530  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.490111 10341 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.444526 10286 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:00:12.509187  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.499769  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:12.525053  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.527400  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:12.495512  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55461.6us
I20260709 13:00:12.519299  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:12.522380  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43918us
I20260709 13:00:12.487677 10256 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.528927  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.531273  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:12.569945  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.495936 10521 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:12.607203  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 116903us
I20260709 13:00:12.491353 10569 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.490020 10510 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:12.613436  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.532208 10260 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:12.612486  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 79581.4us
I20260709 13:00:12.617470  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.617187  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:12.614933  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 95693.2us
I20260709 13:00:12.534116 10386 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:12.621541  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:12.624207  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 260651us
I20260709 13:00:12.626562  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:12.634626  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.523473 10572 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.606232 10485 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.609469 10486 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.616315 10437 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.642804  8824 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.616474 10378 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.612597 10539 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.644479 10539 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:12.645013  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 118004us
I20260709 13:00:12.620695  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.646247  8864 leader_election.cc:400] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:12.649406  8864 leader_election.cc:403] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:00:12.643481  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114641us
I20260709 13:00:12.650703  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:12.631498 10569 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:12.603464 10197 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:00:12.619979 10571 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.637493  8815 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:12.632339 10260 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:12.646986 10386 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:12.652806  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.643309  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.654910  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:12.647300  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 157993us
I20260709 13:00:12.626493 10510 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.657241  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.641927 10485 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.659523  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.645107 10378 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:12.648144 10539 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.631353 10487 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:12.661860  9449 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.666363  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:12.721318  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 86758.5us
I20260709 13:00:12.670960 10584 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.655221  9440 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.624219 10454 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.721498  9449 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:12.670645 10546 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:00:12.721980  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55766.5us
I20260709 13:00:12.664826 10260 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:12.727869  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 68442.3us
I20260709 13:00:12.736244  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.737751  8864 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.728751  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:12.725082  9440 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:12.725165 10584 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:12.661355 10560 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:00:12.739409  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:12.740475  9447 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.711431 10485 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.661646 10256 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.743939  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.666268 10507 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:00:12.708909 10568 raft_consensus.cc:2764] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:00:12.680394 10378 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:12.745846  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 70550.7us
W20260709 13:00:12.759980  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103235us
I20260709 13:00:12.761850  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.762825  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:12.764824  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:12.764750 10546 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.764458 10558 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:12.769071 10576 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:00:12.720854 10557 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:12.769815 10580 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:00:12.770936 10508 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.734706 10554 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:12.762827  9449 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.742082 10571 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:12.797709  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58400.1us
I20260709 13:00:12.724030 10585 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:12.800328  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.804903  9447 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.805003  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:12.746069 10584 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.757402 10481 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:12.766857 10487 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.816079  9449 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.774449  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.767364 10546 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:12.817827  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 191885us
I20260709 13:00:12.803504 10568 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.759199 10486 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:12.825644  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.819294 10487 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:12.846956  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:12.838670 10552 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:12.839953 10579 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.841677 10558 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:12.828174 10554 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.829017 10546 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.847787  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:12.846069  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 220385us
I20260709 13:00:12.838320  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.851938  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:12.852620  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:12.854096  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.855412  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:12.822638 10587 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.832149  9440 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:12.868842 10588 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:12.893785  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 149939us
I20260709 13:00:12.897136  8892 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:12.925518  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58811.2us
I20260709 13:00:12.926589  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:12.849395 10487 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.831784 10508 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:00:12.851217 10579 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.873112 10325 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.926362 10572 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.933051 10189 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.872886 10587 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:12.871614 10529 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:00:12.953043 10588 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:12.957110 10454 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.983872  8889 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:00:13.042551  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 116073us
I20260709 13:00:12.987051 10572 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:13.042832  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 148313us
I20260709 13:00:13.062570  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.070588  9448 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:13.070910  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.032768 10591 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.001428 10587 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.074769  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.064678 10572 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:13.097374  8748 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.640s	user 0.019s	sys 0.011s
I20260709 13:00:13.097818  8748 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:13.095486  9449 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:13.080278  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 228421us
I20260709 13:00:13.099469  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:13.092923 10598 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.100286  9448 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:13.070559 10501 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:12.821441 10569 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:13.063367 10189 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.104678  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.102926  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.113509 10189 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:13.114435 10497 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.009s
I20260709 13:00:13.114828 10497 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.105918 10591 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.116704 10497 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.116731 10591 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:13.117341 10591 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:13.120918 10561 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.720s	user 0.000s	sys 0.007s
W20260709 13:00:13.122185 10537 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.000s
I20260709 13:00:13.105351 10557 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.103796 10501 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.136031  9448 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.136409  9447 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:13.137331  9449 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.138998  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:13.141371  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.142114  8892 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 2: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:13.123366  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1455) took 1140 ms. Trace:
I20260709 13:00:13.134778 10497 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.156267  9447 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.159307 10606 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.159642  9448 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.159873  8897 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.134323 10386 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:13.161227  8897 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:13.161399 10521 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.542s	user 0.023s	sys 0.002s
I20260709 13:00:13.163890  8818 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:13.164314  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.161542 10386 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:13.165565 10604 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:00:13.166671  8897 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.167309  8898 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.107339 10556 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.139981 10554 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.165567  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.171017  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:13.205268  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 143140us
W20260709 13:00:13.218567  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.199683 10598 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:13.219152  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48391.9us
I20260709 13:00:13.219715 10559 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.142124 10424 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:00:13.221971  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.222318  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.223546  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.223781  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.159858 10606 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.171273 10605 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.126602 10600 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:00:13.239681  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:13.171017  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.565s (SENT)
I20260709 13:00:13.227191  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.166911  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.240626  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.240856  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.219959 10519 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:00:13.226008 10425 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.002s
I20260709 13:00:13.199533 10501 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:13.237407  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65322.8us
I20260709 13:00:13.225019 10529 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.260537 10600 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:13.261116  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 456290us
I20260709 13:00:13.171818 10567 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 13:00:13.262301  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 91371us
W20260709 13:00:13.253170 10447 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.006s
I20260709 13:00:13.268658  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1446) took 1301 ms. Trace:
I20260709 13:00:13.269312  8748 rpcz_store.cc:276] 0709 13:00:11.967318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.980157 (+ 12839us) service_pool.cc:224] Handling call
0709 13:00:13.268533 (+1288376us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54003bfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:13.268614 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034691328}
I20260709 13:00:13.132190 10588 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:13.265594 10555 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.010s
I20260709 13:00:13.261886 10378 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.298831 10447 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.238011 10386 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.242141 10537 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:00:13.225916 10554 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.166990 10497 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.259833 10611 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.304863  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.304446 10567 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:00:13.261667 10559 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:13.308569  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65651.1us
I20260709 13:00:13.310177  8741 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.265753 10197 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.263319  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.333411 10556 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.228452 10601 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.243785 10189 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:00:13.260061 10529 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:00:13.305337 10614 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:00:13.258013 10521 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 13:00:13.310834 10260 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.550s	user 0.024s	sys 0.012s
I20260709 13:00:13.303253 10501 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:13.326895  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 99843.8us
I20260709 13:00:13.327908  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.241377 10584 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.323316 10599 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:13.324129 10510 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.600s	user 0.020s	sys 0.006s
I20260709 13:00:13.340194  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:13.309553 10555 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:13.331142 10539 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.606s	user 0.016s	sys 0.049s
I20260709 13:00:13.261240 10600 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.331485 10480 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:13.346191  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.328680 10611 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.232479 10598 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.334746 10606 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.347661 10618 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.358039  8821 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:00:13.350337 10603 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:00:13.349130 10386 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.350325  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.352053 10423 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:13.345865 10260 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.361079  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.157613  8743 rpcz_store.cc:276] 0709 13:00:11.987172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.434091 (+446919us) service_pool.cc:224] Handling call
0709 13:00:12.451732 (+ 17641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:12.744965 (+293233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:12.745027 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:12.807338 (+ 62311us) log.cc:844] Serialized 26 byte log entry
0709 13:00:13.098408 (+291070us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:13.100977 (+  2569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.105923 (+  4946us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:13.106833 (+   910us) raft_consensus.cc:1685] finished
0709 13:00:13.106848 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:13.120550 (+ 13702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16654,"raft.queue_time_us":58696,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":79489,"thread_start_us":445229,"threads_started":2,"wal-append.queue_time_us":245095}
I20260709 13:00:13.350510 10589 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:13.366024 10600 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.365962 10423 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.367671  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1799) took 1043 ms. Trace:
I20260709 13:00:13.368450 10480 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:13.368775 10341 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.837s	user 0.003s	sys 0.006s
I20260709 13:00:13.369418 10378 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.370321 10260 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.357542 10256 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.348958 10325 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:00:13.356853 10597 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:00:13.379142  8824 rpcz_store.cc:276] 0709 13:00:12.324518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.325905 (+  1387us) service_pool.cc:224] Handling call
0709 13:00:12.371226 (+ 45321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:12.643924 (+272698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.367451 (+723527us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:13.367611 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624080768}
I20260709 13:00:13.377394 10589 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.375862  8889 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:13.384089  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.384526  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:13.344440 10601 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:13.355465 10539 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.350775 10510 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.371187 10555 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.391016  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.388866 10481 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:00:13.401940 10480 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.416647 10601 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.416883 10480 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.417263  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.418102 10480 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.418612  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.419227 10480 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:13.419327  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.419580 10510 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.383606 10249 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:00:13.420337  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.389149 10341 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.411075  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.421444  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.421931  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:13.407984  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.412111  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.422446  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.387036 10497 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.414690  8823 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.422497  8748 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:13.389187 10521 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:13.419792  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.420902  8746 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.422901  9442 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.435637  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.436110 10325 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.436710  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.436771 10325 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.425199  8824 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:13.423750 10568 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.591s	user 0.001s	sys 0.006s
I20260709 13:00:13.425832  8822 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:00:13.405265 10485 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.007s
W20260709 13:00:13.428504  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44088.5us
I20260709 13:00:13.427899 10519 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.399019 10567 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:00:13.433528  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.434573  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.391633 10611 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.437371  8740 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.401998 10591 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.388517 10600 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 13:00:13.425726 10279 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:00:13.453159  9443 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:13.454512 10538 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.428778  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:13.438689  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.456589 10584 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.422498 10341 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.457978 10589 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.458776 10487 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.460089 10423 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.450623  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.480183  8746 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:13.462225 10554 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.468639  8744 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.473374 10485 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.478008  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.405362 10572 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.453012 10521 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:13.453032 10613 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:00:13.408797 10546 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.516631 10546 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:13.499591  9414 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.707s (SENT)
I20260709 13:00:13.517510  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.518523 10485 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:13.518854  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.735s (SENT)
I20260709 13:00:13.520521 10572 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.520901 10591 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.483476 10567 consensus_queue.cc:1243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:13.481406  8816 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:13.504194  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 75559.8us
I20260709 13:00:13.505494  8824 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:13.461344 10568 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.473664  8823 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:13.510073  8748 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:13.530704  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.532213  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.479669 10587 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.498471 10564 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:13.498196 10538 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.502743 10611 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.522714  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.523305 10341 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.513195 10521 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.529491 10600 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.539789  8744 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:13.540454  8824 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:13.504923  8743 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.544942  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.529567  8816 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.537681  8893 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.564380  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.565639  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:13.566712  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.567616  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.568514  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.503444 10579 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.554211  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:13.555002 10619 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:00:13.556298  8891 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.557489  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.524446 10325 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.603077 10539 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:13.562597 10521 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.563889  8816 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.582154  8740 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:13.584575  8744 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 5. Ops: []
I20260709 13:00:13.586047  8824 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.591055  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.593538  8746 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.599510  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-31c225a50048451db8964738c5db7540 received from {username='slave'} at 127.0.0.1:36246
I20260709 13:00:13.601748  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 406) took 1071 ms. Trace:
I20260709 13:00:13.593948 10538 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.579378 10197 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.619282  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:13.618010  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 95436.2us
I20260709 13:00:13.620949  8820 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:13.622327  8740 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.579370 10454 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:13.624215  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.648694  8740 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.654979  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:13.645229  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.610663 10568 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:13.634259  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-31c225a50048451db8964738c5db7540 on tablet 31c225a50048451db8964738c5db7540 with peer 112105225cc140459688be5d31dcb071
I20260709 13:00:13.618908 10529 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:00:13.616883 10423 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:00:13.635933  8889 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:13.627120  9441 rpcz_store.cc:276] 0709 13:00:12.530487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.532207 (+  1720us) service_pool.cc:224] Handling call
0709 13:00:13.557262 (+1025055us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:13.601663 (+ 44401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7635200}
I20260709 13:00:13.638119  8746 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:13.638449  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.617576 10539 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.625973 10341 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.668507  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.668848  8740 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:13.641263 10605 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.628767  8816 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.643185  9444 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:13.629426 10600 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.642959 10521 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.675885  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.643105  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:13.687243  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:13.688022  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.681550 10539 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:13.688565  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.689850 10539 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.690403  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.690537  8891 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:13.680213  8744 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.641749 10538 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.645038 10197 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.692319  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.676615  8746 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 1
I20260709 13:00:13.671806 10454 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.693742  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.685853  8740 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.695866  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.695899  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.697081  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:13.698031  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.698925  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:13.700333  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:13.700665  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:13.701978  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.702198  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.702828  8896 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.715479  8896 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:13.694319  8746 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:13.717458  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.718261  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.716404  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.719184 10628 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.719102 10629 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:00:13.721175  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.672755 10568 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.719735  8740 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.697990 10539 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.698417 10605 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.701704  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.716790 10572 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:13.700132 10591 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.691747  8743 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:13.720136  8746 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:13.722371 10552 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:00:13.721679 10617 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:00:13.704429 10571 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.704245 10559 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.699577 10445 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.704030 10606 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.725365 10616 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:00:13.748729 10537 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.749255 10537 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.721859 10632 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:00:13.750540 10632 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.750231 10537 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.717885 10341 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.752439 10559 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.753378 10448 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.728772  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.732635 10572 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:13.698102 10487 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.753510 10559 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.728137  8740 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:13.723104  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.756320 10572 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.758009  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.742455 10571 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.760990  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.730623 10587 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:13.734587  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.730444 10568 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:13.704293 10182 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.731133 10628 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:00:13.736672  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:13.730094 10454 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:13.741014  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:13.732385  8896 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:13.712334 10325 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:00:13.739312  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.742866 10578 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.743638 10519 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.744452  8898 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:13.743664  8895 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:13.747659  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 413) took 1096 ms. Trace:
I20260709 13:00:13.750913 10632 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.753795 10448 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.757614 10537 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.760462  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.759424 10616 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.777202 10448 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.761637 10567 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:13.762494  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.764571  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.765859  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.754182 10286 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.717774 10600 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:13.763234 10571 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.769143  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 249) took 1670 ms. Trace:
I20260709 13:00:13.769243 10568 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:13.769670 10529 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:13.769919 10605 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.775825  8745 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:13.773574 10423 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:13.772648  9442 rpcz_store.cc:276] 0709 13:00:12.654071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.660556 (+  6485us) service_pool.cc:224] Handling call
0709 13:00:12.742503 (+ 81947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.435116 (+692613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.747306 (+312190us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400197654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:13.747579 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555927296}
I20260709 13:00:13.785238  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.810310  8898 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.787047  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:13.787958  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.790803 10567 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.770853 10560 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:00:13.794607  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:13.773631 10501 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:00:13.817651 10437 raft_consensus.cc:993] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: : Instructing follower 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 13:00:13.817927  8895 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 2: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:13.787880 10448 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.798277 10286 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.804247  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:13.799191  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:13.800938  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.773793 10578 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:13.774807  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:13.804709 10568 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:13.791030 10325 consensus_queue.cc:1243] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:00:13.834869  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60142.5us
I20260709 13:00:13.799350 10632 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:13.495895 10612 consensus_queue.cc:1175] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "29c7934ee528454785432732d01ed0a0" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:13.812549  8740 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.775484 10182 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:00:13.810920  9446 raft_consensus.cc:2376] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2. Re-sending same reply.
I20260709 13:00:13.809345  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:13.817493  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:13.840643  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 410) took 1161 ms. Trace:
W20260709 13:00:13.797636  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.820839  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.798983  8743 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:13.804944 10606 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.824648  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:13.825475  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.830706  8898 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:13.809743  9448 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:00:13.835608  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 203) took 1731 ms (client timeout 1707 ms). Trace:
I20260709 13:00:13.831743  8748 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.811902 10567 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:13.810402  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.837051  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.836438 10628 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.834209 10189 raft_consensus.cc:1081] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Signalling peer 7fd110c312c14e17a2bbe6d453493cf2 to start an election
I20260709 13:00:13.831208  9441 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:13.841357  9446 rpcz_store.cc:276] 0709 13:00:12.679229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.786790 (+1107561us) service_pool.cc:224] Handling call
0709 13:00:13.840588 (+ 53798us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:13.843241  8747 rpcz_store.cc:269] 0709 13:00:12.104014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.332182 (+228168us) service_pool.cc:224] Handling call
0709 13:00:13.831442 (+1499260us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:13.835537 (+  4095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200820096}
W20260709 13:00:13.846020  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45189.8us
I20260709 13:00:13.802562  8815 rpcz_store.cc:276] 0709 13:00:12.098896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.099844 (+   948us) service_pool.cc:224] Handling call
0709 13:00:12.238896 (+139052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:12.664746 (+425850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.768893 (+1104147us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400075654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:13.769087 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856544640}
I20260709 13:00:13.848160  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:13.852664  8895 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.837929 10578 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:13.854185 10578 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.856843  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.857419  8819 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.836606  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 413) took 1113 ms. Trace:
I20260709 13:00:13.859592  8744 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:13.857929 10617 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:13.861956 10617 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.863364  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.864840  8746 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:13.865207  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.850140 10341 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:00:13.866853  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:13.867411  8892 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:13.840945  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:13.839485 10423 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:13.850580 10600 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.851866 10560 consensus_queue.cc:1243] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:13.853367 10572 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.876910  8892 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:13.855906 10437 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:13.891153  8895 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:13.892964  9446 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.893738  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.893781  8895 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.895596  8892 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:00:13.874027  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.900139  8892 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.884016  8817 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.904691  9446 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.905731  8895 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:13.908501  8892 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.909488  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:13.901001 10590 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:13.883876 10423 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:13.911445  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.914251  8895 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.914640  9446 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:13.865157 10617 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.899302 10572 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:13.916473  8819 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:13.917878  8892 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.862919  9443 rpcz_store.cc:276] 0709 13:00:12.722525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.855109 (+132584us) service_pool.cc:224] Handling call
0709 13:00:13.831558 (+976449us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400660154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:13.835429 (+  3871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936651648}
I20260709 13:00:13.902391 10437 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:13.923501  9443 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.923849 10437 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.903319 10554 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.905598 10560 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:13.905474  8745 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.919555  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.926733  9446 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.912451  8743 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:13.931152 10634 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:00:13.931363  8895 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.927695  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:13.929152 10572 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.930076  8745 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:13.913947 10286 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:13.945587  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 405) took 1415 ms (client timeout 1738 ms). Trace:
I20260709 13:00:13.934386 10640 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:13.946060  9444 rpcz_store.cc:269] 0709 13:00:12.530166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.532404 (+  2238us) service_pool.cc:224] Handling call
0709 13:00:13.945393 (+1412989us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54001937d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:13.945526 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123514624}
I20260709 13:00:13.912967 10636 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:13.938618  8745 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.938809 10423 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.907907 10568 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:13.918897 10552 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.939536 10586 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.941825 10279 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.938462  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.938437  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1453) took 1960 ms. Trace:
I20260709 13:00:13.950151 10481 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:13.961984  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:13.969133  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.948509  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:14.021636  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.052927  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:13.921275 10600 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:00:14.058522  8749 rpcz_store.cc:276] 0709 13:00:11.978233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.379669 (+401436us) service_pool.cc:224] Handling call
0709 13:00:12.380052 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:12.492984 (+112932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:12.493029 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:12.532567 (+ 39538us) log.cc:844] Serialized 26 byte log entry
0709 13:00:12.741476 (+208909us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:12.765692 (+ 24216us) log.cc:844] Serialized 14 byte log entry
0709 13:00:12.765884 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.367128 (+601244us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b4800080058. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:00:13.692662 (+325534us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b54000b5c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:13.692694 (+    32us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:13.758392 (+ 65698us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b1800167aa0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1
0709 13:00:13.831251 (+ 72859us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800167aa0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:13.938238 (+106987us) raft_consensus.cc:1685] finished
0709 13:00:13.938261 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:13.938383 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18853,"prepare.queue_time_us":18185,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":77894,"raft.run_cpu_time_us":16394,"raft.run_wall_time_us":879188,"spinlock_wait_cycles":275035008,"thread_start_us":100237,"threads_started":1,"wal-append.queue_time_us":212610}
I20260709 13:00:14.067737  8819 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:14.049229 10182 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:13.934868 10560 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.061156 10279 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.060777 10588 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:14.063338 10481 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.076789  8745 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:14.082099  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.077330 10423 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.096374  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1434) took 1259 ms. Trace:
I20260709 13:00:14.097064  8894 rpcz_store.cc:276] 0709 13:00:12.836441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.844838 (+  8397us) service_pool.cc:224] Handling call
0709 13:00:13.826604 (+981766us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:13.826786 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.826998 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:13.902649 (+ 75651us) log.cc:844] Serialized 14 byte log entry
0709 13:00:14.095980 (+193331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.096044 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:14.096082 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.096159 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.096287 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":229484,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":8734592,"thread_start_us":254339,"threads_started":2,"wal-append.queue_time_us":193197}
I20260709 13:00:14.098547  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.099783  8824 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:14.076018 10586 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:14.108623  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 417) took 1288 ms. Trace:
I20260709 13:00:14.106642 10425 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.108682 10640 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.113221  9448 rpcz_store.cc:276] 0709 13:00:12.822540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.586533 (+763993us) service_pool.cc:224] Handling call
0709 13:00:13.587219 (+   686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:13.842992 (+255773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.108293 (+265301us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.108545 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508947200}
W20260709 13:00:14.135355  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.140591  8894 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:14.143410  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 418) took 1472 ms. Trace:
I20260709 13:00:14.100765 10569 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.063776 10552 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:14.143834  9449 rpcz_store.cc:276] 0709 13:00:12.671049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.101499 (+430450us) service_pool.cc:224] Handling call
0709 13:00:13.137084 (+ 35585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.137808 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.143218 (+1005410us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5400661a50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:14.143351 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65718016}
W20260709 13:00:14.146524  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.151453  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:14.115914 10425 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.085215 10182 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.138893 10605 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.140447 10564 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.090911 10588 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:14.169569  8713 leader_election.cc:336] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.015s (SENT)
I20260709 13:00:14.172334  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:14.135878 10586 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.171567  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:14.149006 10643 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:00:14.173018  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:14.158749 10569 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:14.182845  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.183965  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:14.192277  8749 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:14.159708 10539 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.185264  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:14.189668  8862 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe13736604454be3b391b8e946192961. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:14.213574  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:14.186730  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:14.188050  8824 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.217650  9449 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.219465  9447 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.145756 10279 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.231984  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:14.258611  9448 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.260294  9448 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 1 LEARNER]: Advancing to term 3
W20260709 13:00:14.229996  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.180563 10557 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.261752  8740 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.264127  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1457) took 2276 ms. Trace:
W20260709 13:00:14.264706  8818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 252) took 2164 ms (client timeout 1525 ms). Trace:
W20260709 13:00:14.265579  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51333.9us
I20260709 13:00:14.174559 10564 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.187510  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.189530  8890 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.186618 10424 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.190716 10605 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.268514  9447 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:14.267678  9449 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:14.191653 10589 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.218318 10636 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:00:14.301571  8741 rpcz_store.cc:276] 0709 13:00:11.988151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.455547 (+467396us) service_pool.cc:224] Handling call
0709 13:00:12.455830 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.332376 (+876546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.263890 (+931514us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5400465ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.264058 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964704128}
I20260709 13:00:14.323156 10572 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.189122 10437 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.190258 10639 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.191088 10487 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 13:00:14.223517 10501 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:00:14.268424 10448 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:14.256712 10600 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.223598 10538 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:00:14.349088  8824 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:14.221464 10599 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 13:00:14.260609 10189 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:00:14.221438 10539 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.345880  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.303524  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:14.304068  8818 rpcz_store.cc:269] 0709 13:00:12.100613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.102301 (+  1688us) service_pool.cc:224] Handling call
0709 13:00:14.264563 (+2162262us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54000203d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:14.264649 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710036736}
I20260709 13:00:14.444736  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.347520 10279 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.348845  8815 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.322757  8817 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:14.342322 10424 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:14.347502 10589 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:14.348346 10649 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.436990  9448 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 3 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:14.383898 10650 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.440094  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.265751 10425 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.441497  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.442441 10560 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.374244 10557 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 13:00:14.374431  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 161034us
W20260709 13:00:14.446732  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.374347 10605 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.451808  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1432) took 1616 ms. Trace:
I20260709 13:00:14.449791 10647 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.434293 10437 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:14.441107  9441 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:14.461970  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:14.456077  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.423707 10572 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.452684 10645 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:00:14.374902 10569 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.468616  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 166848us
I20260709 13:00:14.453956 10568 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.470644  8824 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:14.477569  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1817) took 1050 ms. Trace:
I20260709 13:00:14.477872  8824 rpcz_store.cc:276] 0709 13:00:13.426668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.459060 (+1032392us) service_pool.cc:224] Handling call
0709 13:00:14.470319 (+ 11259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.477310 (+  6991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.477517 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.459147 10589 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.480727  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:14.486297  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1819) took 1059 ms. Trace:
I20260709 13:00:14.487658  8824 rpcz_store.cc:276] 0709 13:00:13.427200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.485417 (+1058217us) service_pool.cc:224] Handling call
0709 13:00:14.485818 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.485988 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.486011 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.486050 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:14.486077 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.486130 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.486244 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.470079 10425 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.473254 10657 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.460220 10656 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:00:14.494271 10661 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:14.500267 10437 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:14.501397 10651 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:00:14.502689 10586 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:14.502893  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 280) took 1369 ms (client timeout 1516 ms). Trace:
W20260709 13:00:14.503363  8889 rpcz_store.cc:269] 0709 13:00:13.133176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.140481 (+  7305us) service_pool.cc:224] Handling call
0709 13:00:14.502704 (+1362223us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:14.502833 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262831488}
I20260709 13:00:14.503954  8824 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:14.504503  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1820) took 1077 ms. Trace:
I20260709 13:00:14.504972 10539 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.505551  8898 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:14.507884  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 284) took 1354 ms. Trace:
I20260709 13:00:14.508266  8893 rpcz_store.cc:276] 0709 13:00:13.153061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.156819 (+  3758us) service_pool.cc:224] Handling call
0709 13:00:13.159321 (+  2502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.564703 (+405382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.507650 (+942947us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54000aafd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.507822 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833789184}
I20260709 13:00:14.487033 10568 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.472142 10182 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:14.474558 10647 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.529237 10589 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.530247 10647 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:14.511927 10568 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.495594 10650 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:14.532838 10650 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.534961 10650 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:14.512605  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.496696  8817 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:14.512830 10652 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:00:14.535970  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1818) took 1109 ms. Trace:
I20260709 13:00:14.513650 10564 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.536300  8817 rpcz_store.cc:276] 0709 13:00:13.426903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.468482 (+1041579us) service_pool.cc:224] Handling call
0709 13:00:14.493997 (+ 25515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.535721 (+ 41724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.535916 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.538679  8817 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:14.539001  8748 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:14.510758  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.513747  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.514616 10658 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.540011  9444 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:14.496933  9443 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.540191 10658 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.540815  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.540992  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.477348 10588 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:14.496971 10653 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:00:14.515446  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:14.547307  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.516862 10649 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:00:14.517397 10571 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.711s	user 0.006s	sys 0.007s
I20260709 13:00:14.549203 10649 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.549121 10571 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.545841 10628 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:14.550583 10628 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:14.550915 10628 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.551267 10571 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:14.518831  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:14.551721 10628 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.552758  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:14.552980 10587 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:14.467614  8890 rpcz_store.cc:276] 0709 13:00:12.835892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.845151 (+  9259us) service_pool.cc:224] Handling call
0709 13:00:13.942267 (+1097116us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:13.942470 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.342159 (+399689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.451517 (+109358us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54000492d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.451742 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533309056}
I20260709 13:00:14.553628  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1151) took 1332 ms. Trace:
I20260709 13:00:14.519549 10659 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.554442  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:14.520056 10660 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:14.555277 10660 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.555487  8890 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:14.523567  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:14.504781  8824 rpcz_store.cc:276] 0709 13:00:13.427462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.488895 (+1061433us) service_pool.cc:224] Handling call
0709 13:00:14.503624 (+ 14729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.504292 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.504452 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.524293  8819 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.497160 10657 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.525710 10539 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.527113  8745 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:14.571668  8745 pending_rounds.cc:85] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 1
W20260709 13:00:14.571724  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.572026  8745 pending_rounds.cc:107] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:14.572289  8745 raft_consensus.cc:2889] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:14.534871 10589 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:14.573489 10589 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:14.574136 10589 consensus_queue.cc:1243] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:14.574565 10589 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:14.574811  8748 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:14.575886 10589 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:14.576537  8748 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:14.576902 10454 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:00:14.577355 10539 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.577905 10632 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.578352  8748 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:14.578784  8748 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.579535 10539 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:14.579663  8748 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.581103  8748 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.513948 10197 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.553486 10587 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.522575 10447 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:14.552883  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 604738us
I20260709 13:00:14.519250 10572 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.554041  8823 rpcz_store.cc:276] 0709 13:00:13.221629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.221752 (+   123us) service_pool.cc:224] Handling call
0709 13:00:14.553461 (+1331709us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400083ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:14.553564 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409682176}
I20260709 13:00:14.554455  8892 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.559113  8824 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.562101  9442 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:14.561003 10588 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:00:14.554401 10659 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.561225 10661 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:14.557034 10658 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.556136 10660 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.564179  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:14.565721  8819 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:14.565743  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.613169  8819 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:14.566488 10481 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.566802 10657 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:14.622469  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.622788 10657 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.623736  8815 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:14.624408 10659 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:14.622484 10481 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.625571  8817 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:14.610744  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.567863 10567 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.568732 10611 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:14.637235 10611 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.637857  8815 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.637452  8819 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.638621  8819 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.638226 10611 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.571237  8891 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:14.583714  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.552942 10628 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.594630  8647 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 2 to 4, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:13.592168  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:14.596892  8824 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:14.651458  8824 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:14.598098  8823 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:14.588228 10587 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.599756 10661 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.621814  8748 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:00:14.659255  8748 pending_rounds.cc:85] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 4
I20260709 13:00:14.659578  8748 pending_rounds.cc:107] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:00:14.659812  8748 raft_consensus.cc:2889] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:14.642831  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.658628 10189 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:00:14.665926  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:14.666507  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:14.666801 10571 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:14.667281 10571 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.667591  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:14.639271  8815 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.669265 10571 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.669435 10286 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:14.672008  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:14.674594  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.673733  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.624953 10659 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.675875  8823 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:14.639248 10611 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.676256  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.674676  8890 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:14.681814  8748 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:14.686374  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.641556 10658 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:14.686437  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:14.681934  8819 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:14.666530  8891 raft_consensus.cc:2376] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4. Re-sending same reply.
I20260709 13:00:14.640074 10578 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 13:00:14.639911 10656 consensus_queue.cc:1175] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 2 status { last_received { term: 1 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:14.679711 10569 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.694511 10572 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.685010 10587 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.672796 10539 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:14.684816  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.709939  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:14.711369  8818 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:14.708112 10572 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.706207 10448 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.716701 10572 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.717679 10448 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.718441 10519 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.718858 10519 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.719748 10519 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.721665 10480 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:00:14.722436 10598 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.706516 10600 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:00:14.723505  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1160) took 1499 ms. Trace:
I20260709 13:00:14.724043  8821 rpcz_store.cc:276] 0709 13:00:13.224430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.244246 (+ 19816us) service_pool.cc:224] Handling call
0709 13:00:13.264691 (+ 20445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:13.361828 (+ 97137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.723050 (+1361222us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b54002297d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.723409 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206617856}
I20260709 13:00:14.709944 10569 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.726344  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.727157  8742 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:14.727725 10569 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.728263  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.744382 10612 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.730844  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1443) took 1002 ms. Trace:
I20260709 13:00:14.744923 10612 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:14.736936  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:14.761792  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:14.737577 10661 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.692806 10182 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:00:14.766418 10552 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:14.740231  9440 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.768735  8895 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:14.771790  8643 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 298) took 9154 ms. Trace:
I20260709 13:00:14.740506  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:14.741144  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 215) took 1270 ms. Trace:
I20260709 13:00:14.742215  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:14.742851 10662 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:14.773537  8740 rpcz_store.cc:276] 0709 13:00:13.471147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.839733 (+368586us) service_pool.cc:224] Handling call
0709 13:00:13.840030 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.301184 (+461154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.740918 (+439734us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54004655d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.741089 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969344896}
I20260709 13:00:14.744943  8896 rpcz_store.cc:276] 0709 13:00:13.727993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.950371 (+222378us) service_pool.cc:224] Handling call
0709 13:00:13.950669 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.065600 (+114931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.065642 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.065688 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:14.065731 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.730556 (+664825us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540046e454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.730617 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.730763 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263955840}
I20260709 13:00:14.774654  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.775194  8896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:50482
I20260709 13:00:14.775662  8896 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:14.775981  8896 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.776034  8740 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:14.777501 10480 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:14.777751 10666 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:14.778178 10480 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:14.778528 10666 consensus_queue.cc:1243] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:14.779551 10666 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:14.779346 10480 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.780423 10666 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:14.782007  8890 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:14.782215 10454 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:14.783502  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1446) took 1046 ms. Trace:
I20260709 13:00:14.747179 10567 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.747558  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:14.748448  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.749464  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.781659  8647 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 2 to 3, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 3 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:14.791553  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.791929  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.749747 10639 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:14.792934  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 100754us
W20260709 13:00:14.793639  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 233429us
I20260709 13:00:14.750182  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.751835 10659 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:14.752535 10646 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.323s	user 0.001s	sys 0.001s
I20260709 13:00:14.812316  8647 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 2 to 4, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 4 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:14.722476  8891 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:14.753183 10425 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:14.754599  9445 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.695845 10658 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.783866  8894 rpcz_store.cc:276] 0709 13:00:13.736989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.149272 (+412283us) service_pool.cc:224] Handling call
0709 13:00:14.153193 (+  3921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.215503 (+ 62310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.215553 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.215610 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:14.215676 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.783294 (+567618us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b540015de54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.783339 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.783439 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142773120}
I20260709 13:00:14.774276 10662 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.809768 10501 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.793108 10539 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:00:14.810972  8821 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:14.796108  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:14.826653  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:14.827432  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:14.829365  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:14.793715 10569 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.839385  8823 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:14.840183  8742 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.841583  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 429) took 1086 ms. Trace:
I20260709 13:00:14.821512 10601 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.762085 10612 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.842015  9443 rpcz_store.cc:276] 0709 13:00:13.755339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.924369 (+169030us) service_pool.cc:224] Handling call
0709 13:00:13.924647 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.496584 (+571937us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b54005b4620. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d1d87 00007f90f64d2470 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:00:14.540805 (+ 44221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.830990 (+290185us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.841513 (+ 10523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59353856}
I20260709 13:00:14.842396 10601 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:14.844941  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.845808 10486 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.846534 10663 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.846962 10663 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.847285 10663 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.772178  8643 rpcz_store.cc:276] 0709 13:00:05.647422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.650301 (+  2879us) service_pool.cc:224] Handling call
0709 13:00:06.145332 (+495031us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:00:14.771714 (+8626382us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:11.605626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:11.793601 (+187975us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:11.793644 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:00:11.793663 (+    19us) tablet.cc:662] Decoding operations
0709 13:00:11.814338 (+ 20675us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:11.832463 (+ 18125us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:11.844923 (+ 12460us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:00:11.890508 (+ 45585us) tablet.cc:701] Row locks acquired
0709 13:00:11.892710 (+  2202us) write_op.cc:260] PREPARE: finished
0709 13:00:11.907399 (+ 14689us) write_op.cc:270] Start()
0709 13:00:11.929835 (+ 22436us) write_op.cc:276] Timestamp: P: 1783602011907345 usec, L: 0
0709 13:00:11.934664 (+  4829us) op_driver.cc:348] REPLICATION: starting
0709 13:00:11.995542 (+ 60878us) log.cc:844] Serialized 924 byte log entry
0709 13:00:12.467491 (+471949us) op_driver.cc:464] REPLICATION: finished
0709 13:00:12.741826 (+274335us) write_op.cc:301] APPLY: starting
0709 13:00:12.746653 (+  4827us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:12.766623 (+ 19970us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:12.827612 (+ 60989us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:12.943687 (+116075us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:12.992547 (+ 48860us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:00:13.028233 (+ 35686us) write_op.cc:312] APPLY: finished
0709 13:00:13.064488 (+ 36255us) log.cc:844] Serialized 29 byte log entry
0709 13:00:13.095288 (+ 30800us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:13.099525 (+  4237us) write_op.cc:454] Released schema lock
0709 13:00:13.103142 (+  3617us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:13.134302 (+ 31160us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:13.138112 (+  3810us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2275,"reported_tablets":7,"spinlock_wait_cycles":23468928,"child_traces":[["op",{"apply.queue_time_us":255952,"apply.run_cpu_time_us":3091,"apply.run_wall_time_us":393529,"num_ops":2,"prepare.queue_time_us":467394,"prepare.run_cpu_time_us":39864,"prepare.run_wall_time_us":497475,"raft.queue_time_us":147813,"raft.run_cpu_time_us":72023,"raft.run_wall_time_us":186171,"replication_time_us":532685,"thread_start_us":827526,"threads_started":4,"wal-append.queue_time_us":270522,"wal-append.run_cpu_time_us":48368,"wal-append.run_wall_time_us":1845283}]]}
I20260709 13:00:14.819469  8894 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.798099  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:14.792821 10639 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:14.821341  8647 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 1 to 3, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 3 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:14.823447  8741 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:14.846278 10486 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.855268 10601 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:14.812933  8896 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:14.830742  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:14.849227  8890 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:14.838145 10598 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.823211  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1452) took 1025 ms. Trace:
I20260709 13:00:14.841099 10569 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.872941  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:14.855060  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1453) took 1030 ms. Trace:
I20260709 13:00:14.873498  8742 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:14.877753  8896 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:14.836678 10632 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:00:14.837349 10501 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.900946  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 191178us
I20260709 13:00:14.910091  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:14.874775 10569 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.902379 10648 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:14.900573 10628 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.909578  9440 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.876237 10598 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:14.948035  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.961726  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.966516  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.966929  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.969908  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1157) took 1746 ms (client timeout 2024 ms). Trace:
W20260709 13:00:14.970398  8820 rpcz_store.cc:269] 0709 13:00:13.223461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.223582 (+   121us) service_pool.cc:224] Handling call
0709 13:00:14.944486 (+1720904us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:14.969842 (+ 25356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881624320}
W20260709 13:00:14.971463  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.876101  8891 rpcz_store.cc:276] 0709 13:00:13.825077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.721939 (+896862us) service_pool.cc:224] Handling call
0709 13:00:14.722269 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.818386 (+ 96117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.849786 (+ 31400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.924207 10673 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:14.913204 10639 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.002393 10675 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:15.003803  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:14.984779 10570 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:00:15.000626 10598 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:14.941797 10286 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:00:14.996579  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 409) took 2454 ms. Trace:
I20260709 13:00:14.940972 10628 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:14.874540  8893 rpcz_store.cc:276] 0709 13:00:13.795207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.509222 (+714015us) service_pool.cc:224] Handling call
0709 13:00:14.509545 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.509857 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.509892 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.510352 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.680673 (+170321us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:14.687977 (+  7304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.702905 (+ 14928us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:14.798633 (+ 95728us) raft_consensus.cc:1685] finished
0709 13:00:14.812056 (+ 13423us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.820578 (+  8522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70823,"raft.queue_time_us":4559,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":131198,"threads_started":1,"wal-append.queue_time_us":193844}
I20260709 13:00:14.995707 10680 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.933943 10648 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.022082 10648 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:15.023696  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:15.006799 10676 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.002924 10675 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.016161 10501 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.017351 10680 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.016261 10661 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:15.041895  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.027746 10537 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:14.997378 10673 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.030360 10662 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:15.036088  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.060084  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.047700 10680 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.047989 10537 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:15.049121 10486 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:15.062283 10682 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:00:15.011469  9440 rpcz_store.cc:276] 0709 13:00:12.531441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.607296 (+ 75855us) service_pool.cc:224] Handling call
0709 13:00:12.652057 (+ 44761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.739746 (+2087689us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b58005f03d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:00:14.940330 (+200584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.984365 (+ 44035us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595198848}
I20260709 13:00:15.030062 10676 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.081360 10286 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:15.072499  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.092617  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:15.103291  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.104476  8820 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:15.104732  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.046690 10661 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.112362  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:15.099646 10669 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:00:15.084055 10649 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.114897 10564 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:15.117733  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 207821us
W20260709 13:00:15.101540 10672 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:00:15.102303 10378 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.115424 10648 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.133824 10579 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.116776  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.116659 10662 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.113621 10650 raft_consensus.cc:2764] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:15.141722  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.142401  9440 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:15.142524 10675 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:15.143136 10598 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:15.141765  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:15.145805  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.145710  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:15.146229  8742 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:15.144007  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.131832 10378 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.105913 10286 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.125674 10649 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.137790 10648 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.156292  8821 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:15.159158  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.155829 10182 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:15.159852  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.160960 10378 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:15.162950 10642 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.052s	user 0.005s	sys 0.000s
I20260709 13:00:15.163944  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1442) took 1440 ms. Trace:
I20260709 13:00:15.137153 10572 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.164171  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1450) took 1396 ms. Trace:
I20260709 13:00:15.164317  8897 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.174167  8892 rpcz_store.cc:276] 0709 13:00:13.723357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.924685 (+201328us) service_pool.cc:224] Handling call
0709 13:00:13.925407 (+   722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.589931 (+664524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.163693 (+573762us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:15.163878 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444921728}
W20260709 13:00:15.175599  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.156270  9444 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:15.176805  8892 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:15.177110  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1451) took 1390 ms. Trace:
I20260709 13:00:15.154701  8891 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:15.182507  8891 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.182834 10557 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.183782  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.132228 10537 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.165457 10605 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.169008  9440 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:15.170120  8746 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:15.168964 10628 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.173360  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:15.193862  8815 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:15.177520  8898 rpcz_store.cc:276] 0709 13:00:13.786407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.506827 (+720420us) service_pool.cc:224] Handling call
0709 13:00:14.507091 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.507369 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.507392 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.507823 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.692862 (+185039us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:14.692901 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.692963 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:15.176902 (+483939us) raft_consensus.cc:1685] finished
0709 13:00:15.176926 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.177054 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3625,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":184677,"threads_started":1,"wal-append.queue_time_us":352705}
I20260709 13:00:15.152105 10662 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:15.154428  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.197739  9441 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:15.198029 10572 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.198855  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.200276  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.173929 10649 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.174470 10182 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:15.184484  8821 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:15.164966 10424 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.151141 10673 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.190042 10661 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.195276  8893 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.194227  8815 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:15.196919 10688 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:00:15.210206  8748 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:15.183353 10557 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:15.200945  9441 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:15.191022  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1821) took 1493 ms. Trace:
I20260709 13:00:15.207226 10537 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:15.224890 10378 consensus_queue.cc:1175] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:15.211241 10182 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:15.238332 10182 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.210820 10605 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:00:15.191613 10538 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:00:15.240599  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.222290 10424 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.243861  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.236222  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:15.215811 10572 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:00:15.248975  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.250068  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.252895  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.255421  8820 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:15.256758  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.244375 10605 raft_consensus.cc:476] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:00:15.254232 10650 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:15.229116  8817 rpcz_store.cc:276] 0709 13:00:13.693951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.539367 (+845416us) service_pool.cc:224] Handling call
0709 13:00:14.625365 (+ 85998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.626089 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.176353 (+550264us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b540028b750. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:15.187592 (+ 11239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197132672}
I20260709 13:00:15.259097 10650 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.174671  8889 rpcz_store.cc:276] 0709 13:00:13.768100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.504286 (+736186us) service_pool.cc:224] Handling call
0709 13:00:14.504589 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.504966 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.505002 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.542175 (+ 37173us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.708145 (+165970us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:14.712783 (+  4638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.714759 (+  1976us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:15.163901 (+449142us) raft_consensus.cc:1685] finished
0709 13:00:15.163941 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.164102 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":756,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":152537,"threads_started":1,"wal-append.queue_time_us":318848}
I20260709 13:00:15.258577  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:15.258416 10697 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:00:15.254822 10648 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.270004  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.265105 10605 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.255733  8815 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.325176 10570 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.250541 10557 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.275602 10554 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.338733 10686 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:15.339341  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.273952 10286 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.268141 10569 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.279678 10481 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:15.353739  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.257419 10662 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.355930 10579 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:15.371388 10693 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:00:15.348232 10692 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:00:15.375856 10695 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:00:15.343160  8815 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.375021 10557 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.350134  8821 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:15.347990 10570 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:15.377345 10663 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.001s
I20260709 13:00:15.367736 10579 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.400321  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.398584 10538 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.373685 10481 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.399093 10675 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:15.405210 10660 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.444500  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.404310 10480 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:15.447252 10480 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.445739  8746 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:15.420028 10554 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.454202  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:15.418673 10605 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:00:15.486971 10569 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:15.488512  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.457672 10698 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:00:15.489403 10691 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:00:15.458093  8743 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:15.465788 10700 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:00:15.450590 10675 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.445525 10650 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.456159 10570 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:15.377878 10686 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:15.547691  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.420207 10663 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:15.495674 10480 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.532481 10696 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.551240 10650 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:15.550998 10507 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.539027 10703 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.443457 10538 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.555366  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:15.545399 10570 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.561226  8742 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:00:15.562065  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.558260 10696 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.563110 10650 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:15.554224 10567 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.003s
I20260709 13:00:15.570629  8746 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.568002 10507 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.577272 10570 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.583868 10696 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:15.584359 10507 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.584682 10570 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:15.589598  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.595062  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.594535  8746 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:15.573091 10703 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.588929  8815 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:15.594245 10686 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:15.627161  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.634735  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.638168  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.623756  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.633055 10650 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:15.633644  8824 raft_consensus.cc:606] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:15.646123  8824 raft_consensus.cc:614] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:15.644501  8744 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.647490  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.647810  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.648056  8744 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:15.648694  8745 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:15.649312  8744 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:15.642409 10680 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.003s
W20260709 13:00:15.652901  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.653628  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.654099  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.652256 10537 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.664013  8822 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:15.666013  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.666754  8819 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 2: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:15.667455  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.668078  8819 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:15.668296 10675 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.669461  9446 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:15.659981 10579 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.673728  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.674099 10663 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.674744 10579 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:15.675415 10487 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.676187 10537 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.676654 10487 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.677039 10487 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:15.678560 10665 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.745s	user 0.004s	sys 0.005s
I20260709 13:00:15.678743 10537 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.679400 10538 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.672282 10703 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.650290 10680 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.682943  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1483) took 1766 ms. Trace:
I20260709 13:00:15.683189 10680 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.683426  8749 rpcz_store.cc:276] 0709 13:00:13.916453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.221017 (+304564us) service_pool.cc:224] Handling call
0709 13:00:14.259997 (+ 38980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.372794 (+112797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.372823 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.373342 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.712698 (+339356us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:14.712753 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.712822 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:15.677915 (+965093us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:15.682594 (+  4679us) raft_consensus.cc:1685] finished
0709 13:00:15.682622 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.682875 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6872,"raft.queue_time_us":174489,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9085,"spinlock_wait_cycles":4740480,"thread_start_us":397898,"threads_started":2,"wal-append.queue_time_us":516486}
I20260709 13:00:15.686264 10680 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.682070  9448 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:15.686741 10680 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:15.688768  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.689049 10519 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.689842  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1778) took 2081 ms. Trace:
I20260709 13:00:15.689683 10487 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.690243  8897 rpcz_store.cc:276] 0709 13:00:13.608224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.608346 (+   122us) service_pool.cc:224] Handling call
0709 13:00:14.738120 (+1129774us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:15.164116 (+425996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.180322 (+ 16206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.689654 (+509332us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000483d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:15.689788 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987914368}
I20260709 13:00:15.691794 10487 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.682030 10197 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.692782  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1485) took 1774 ms. Trace:
I20260709 13:00:15.693260 10680 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.696440  8892 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:15.697877  8897 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.674126 10279 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:15.699288 10279 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:15.700107 10279 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.700764  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:15.701197 10279 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.701645  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:15.702576 10481 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.702674  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:15.703541 10559 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.704402 10713 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.704746 10596 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.704882 10559 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.705163 10596 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:15.705681 10559 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.706168 10559 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.706483 10559 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.708763  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.709705 10325 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:15.710247  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.710966 10510 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.711194 10325 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:15.711418 10510 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.712136 10606 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:15.712499 10606 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:15.712841 10606 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.711637  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.714540 10510 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.714794  9442 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:15.705647 10596 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.717964 10676 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.719717  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.720505 10650 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.721077 10650 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:15.721830  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1815) took 2295 ms. Trace:
I20260709 13:00:15.722002 10650 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.722234  8818 rpcz_store.cc:276] 0709 13:00:13.426227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.449863 (+1023636us) service_pool.cc:224] Handling call
0709 13:00:14.460159 (+ 10296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.712451 (+252292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.721587 (+1009136us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540004ddd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:15.721778 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577326848}
I20260709 13:00:15.693164  8743 rpcz_store.cc:276] 0709 13:00:13.918429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.374059 (+455630us) service_pool.cc:224] Handling call
0709 13:00:14.437279 (+ 63220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.490925 (+1053646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.692550 (+201625us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540016e9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:15.692725 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2324788480}
W20260709 13:00:15.726419  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.727753  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.729592  8818 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:15.730168  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.733062  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.735702 10589 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:15.737763  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.735837  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.738339  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:15.738765  8889 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:15.737121  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.740382  8897 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:15.729612 10510 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:00:15.741633  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:15.742177  8893 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.745718  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:15.746513  9442 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.746748 10677 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.747236 10677 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.751071  8893 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:15.761258  8748 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:15.769778  8744 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.757143  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.770659  8741 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:15.769568  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.771795  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.743968 10559 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.774998  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:15.778162  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.772511  8741 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:15.786231  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1474) took 2059 ms. Trace:
I20260709 13:00:15.777545  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.777248  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:15.819442  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.821973  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:15.822284  8744 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:15.822702  8741 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:15.786794  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:15.805524  8818 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:15.826960  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.823487  8893 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:15.821003  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:15.821722  8747 rpcz_store.cc:276] 0709 13:00:13.726955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.874145 (+147190us) service_pool.cc:224] Handling call
0709 13:00:13.885241 (+ 11096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.885474 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:13.907155 (+ 21681us) log.cc:844] Serialized 14 byte log entry
0709 13:00:15.162467 (+1255312us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:15.769347 (+606880us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b18003bf320. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:00:15.785900 (+ 16553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.785955 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:15.785998 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.786068 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.786180 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":326,"prepare.queue_time_us":22003,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":485,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":20631,"spinlock_wait_cycles":13347840,"thread_start_us":15665,"threads_started":1,"wal-append.queue_time_us":177761}
I20260709 13:00:15.828464 10579 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:15.836005  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.832788  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:15.838148  8816 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.834908 10424 raft_consensus.cc:2764] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:15.828749 10661 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.774073 10481 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:00:15.786087 10676 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:15.841245 10661 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.841887 10676 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.843735 10661 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:15.843816  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.844058 10676 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.786525 10557 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:15.837003 10487 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.844844 10557 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.845016 10487 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.847728 10487 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:15.849442  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:15.850262  8818 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:15.862725 10662 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.869201  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:15.774659 10703 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:15.853353  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:15.825559 10538 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:00:15.825906 10713 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:00:15.824383 10673 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:00:15.860028 10510 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.817010 10606 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.873030 10703 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.891978 10510 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.883250  8817 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:15.901805  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.875236 10378 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:00:15.899288 10510 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.872733 10662 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.898007 10703 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.950726 10487 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.952268 10487 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.954491 10606 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:15.954967 10605 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.927928 10378 consensus_queue.cc:1243] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:15.955593 10605 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.927472 10537 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.917733 10661 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:15.918174 10564 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.928781 10677 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:15.978556  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:16.006002  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.976455 10564 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:15.965160 10537 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.009603 10537 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.917910 10596 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:15.971262  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 410) took 3358 ms. Trace:
I20260709 13:00:15.970927 10628 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.005759 10554 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.042028  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.975113 10661 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.037629 10554 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:16.062063 10554 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.062873 10554 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.063503 10554 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.028741  8816 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:16.067397  9442 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
W20260709 13:00:16.071305  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.041688 10656 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.044327 10605 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.048457 10559 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.111099 10586 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.025048  9445 rpcz_store.cc:276] 0709 13:00:12.630662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.630814 (+   152us) service_pool.cc:224] Handling call
0709 13:00:13.597411 (+966597us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:13.597625 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.754273 (+1156648us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b54006642a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d11c0 00007f90f64d2231 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:00:14.820473 (+ 66200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.970934 (+1150461us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400663ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:15.971175 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572757120}
I20260709 13:00:16.040733 10628 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.111357 10658 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.111670  8817 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:16.124426  9447 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:16.113494 10656 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:16.131462 10726 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:00:16.118918 10586 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.135388 10564 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.136960  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44786 (request call id 16) took 1113 ms. Trace:
I20260709 13:00:16.141273 10703 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.142439 10617 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.140239  8824 rpcz_store.cc:276] 0709 13:00:15.023421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.023585 (+   164us) service_pool.cc:224] Handling call
0709 13:00:16.136770 (+1113185us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64c9853 00007f90fb49760a 00007f90f4a1d7d4 00007f90f4a1d762 00007f90f4a1d691 00007f90f4a1d60c 00007f90f4a1c8c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:16.136879 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401573248}
I20260709 13:00:16.147492  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1457) took 2240 ms. Trace:
W20260709 13:00:16.143617 10727 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:00:16.152309 10596 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.148005  8895 rpcz_store.cc:276] 0709 13:00:13.906883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.769852 (+862969us) service_pool.cc:224] Handling call
0709 13:00:14.770179 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.786821 (+ 16642us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:14.943364 (+156543us) log.cc:844] Serialized 14 byte log entry
0709 13:00:16.146806 (+1203442us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b18003dd920. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:00:16.147044 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.147108 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:16.147169 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.147270 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.147420 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":236496,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"spinlock_wait_cycles":19530496,"thread_start_us":213612,"threads_started":2,"wal-append.queue_time_us":162747}
I20260709 13:00:16.143381 10703 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.134261 10510 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:16.166219  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 220489us
W20260709 13:00:16.171304  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 301147us
I20260709 13:00:16.173096  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.184664  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.185726  8824 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:16.186729  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.188211  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.191699  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.193455  8824 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:00:16.239068  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 137243us
W20260709 13:00:16.240928  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.190603 10554 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.179051 10703 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:00:16.234193 10606 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.235150 10586 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.235765  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.243278 10661 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.243093 10656 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.277640 10703 consensus_queue.cc:1243] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:16.289974 10538 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.290928 10736 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:00:16.290810 10538 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.276022 10586 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.292696  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.292651 10586 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.293371  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.295049 10714 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.295549 10735 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:16.300644  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.300724 10596 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:16.308874  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.309859  8817 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:16.310712  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.312008 10510 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.312774  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.313998  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.314419 10559 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.314713  8817 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:16.314939 10559 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.315518  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.316078  8817 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:16.316903  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.259220 10606 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.317513 10731 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.317998 10731 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.276861 10537 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.318943  8819 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:16.319919 10731 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.320591 10537 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.322069 10586 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.324841 10703 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.325341 10703 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:16.326130 10703 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.326390  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.330569  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.331344  8822 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:16.332652  8822 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:16.339955  8823 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:16.341413 10649 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:16.341894 10649 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:16.342888 10649 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.344393 10649 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.349993 10628 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.323876 10510 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.352362 10510 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.354847 10510 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.301568 10714 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:16.357719 10714 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.358980 10714 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.359272 10662 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.361917  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.318970 10606 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.365587 10639 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.324847 10538 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:16.376482  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 599580us
I20260709 13:00:16.378310  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:16.380231  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 603035us
W20260709 13:00:16.382019  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.383030  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:16.383381 10557 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.383996 10286 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.384585 10676 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:16.385922  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.401455  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:16.386390 10197 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.386615 10519 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.387349  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 491) took 1875 ms (client timeout 2054 ms). Trace:
I20260709 13:00:16.402861  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:16.405128  9447 rpcz_store.cc:269] 0709 13:00:14.512171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.512330 (+   159us) service_pool.cc:224] Handling call
0709 13:00:16.387186 (+1874856us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400660dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:16.387296 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3655312256}
I20260709 13:00:16.384033 10680 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:00:16.389254  9443 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:16.387283 10521 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:16.389909 10571 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:16.392693  8740 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:16.413887  8740 pending_rounds.cc:85] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 2
I20260709 13:00:16.414435  8740 pending_rounds.cc:107] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:16.414732  8740 raft_consensus.cc:2889] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:16.397876  8824 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:00:16.189153 10601 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.134s
I20260709 13:00:16.406679  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:16.410218  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:16.411943  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:16.421451  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:16.379588  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:16.403609 10628 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.396171  8817 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:16.403961 10617 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.399883 10286 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.438100  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:16.454608  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.459647  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:16.411691 10197 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.404102 10612 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.413744 10279 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:16.404280 10564 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:00:16.415867 10676 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:16.399718 10662 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:00:16.423316 10587 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.413455 10591 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.458504 10677 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.411145 10658 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.462831 10425 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.425776 10650 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:16.464457 10286 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.453596 10632 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:16.453948 10628 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.477445 10591 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.465162 10741 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.464241 10501 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:00:16.467056 10197 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.422053 10521 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.470434 10279 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.468022  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.415000 10519 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.515586 10279 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:16.515800  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 47911.2us
I20260709 13:00:16.517048 10519 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.517181 10661 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:16.517810 10661 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:16.518890  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1461) took 1987 ms. Trace:
I20260709 13:00:16.519400  8894 rpcz_store.cc:276] 0709 13:00:14.531379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.853745 (+322366us) service_pool.cc:224] Handling call
0709 13:00:14.854127 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.997398 (+143271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.997441 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.670674 (+673233us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:00:15.671165 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:00:15.866671 (+195506us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:15.866709 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.518496 (+651787us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:16.518561 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.518619 (+    58us) raft_consensus.cc:1685] finished
0709 13:00:16.518639 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.518820 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":967,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6046,"spinlock_wait_cycles":343791744,"thread_start_us":195104,"threads_started":1,"wal-append.queue_time_us":289821}
I20260709 13:00:16.518805 10661 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.522449  8891 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:16.474773  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 185244us
I20260709 13:00:16.464287 10617 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.527098 10617 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.475579 10605 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:00:16.473930 10554 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:00:16.480998 10650 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.494558 10632 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.533579 10650 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:16.533739 10632 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.480499 10675 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.468313 10559 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.494534 10677 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.498708 10739 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:00:16.494158 10658 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.540186 10677 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.540369 10658 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.536899 10676 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.543143 10557 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.543663 10557 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.546563  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1490) took 1889 ms. Trace:
I20260709 13:00:16.546943  8740 rpcz_store.cc:276] 0709 13:00:14.657674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.776736 (+119062us) service_pool.cc:224] Handling call
0709 13:00:14.776973 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.415210 (+1638237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.546323 (+131113us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540013bfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:16.546507 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3716761600}
I20260709 13:00:16.548218 10741 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.549024 10741 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.549608 10501 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:16.549577 10741 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.551518 10501 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:16.552299 10741 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:16.552651  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 152421us
W20260709 13:00:16.546180  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.557370  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.558571  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.559365  8740 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:16.498764 10628 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.562459  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.494818  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:16.589895  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:16.565611 10743 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:00:16.508870  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 436) took 1316 ms. Trace:
I20260709 13:00:16.600665 10677 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.601164 10677 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.601317 10737 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:16.601578 10677 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.601788 10737 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:16.602676  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 294) took 2004 ms. Trace:
I20260709 13:00:16.603076  8891 rpcz_store.cc:276] 0709 13:00:14.598060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.183343 (+585283us) service_pool.cc:224] Handling call
0709 13:00:16.522191 (+1338848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.522991 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.602466 (+ 79475us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400248e50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:16.602619 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3062004992}
I20260709 13:00:16.604663 10182 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.605564 10600 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.606040 10600 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.606427 10650 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.608142 10600 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.608539 10650 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:16.599977  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46691us
W20260709 13:00:16.609176  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:16.609679  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.610962  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.608767 10714 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.616061  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.478348 10587 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.575968 10739 consensus_queue.cc:1243] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:16.621687  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.598136 10554 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.643003  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.606431  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 430) took 1698 ms. Trace:
I20260709 13:00:16.670895  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:16.672482  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:16.674965  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:16.637583 10600 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:16.612505  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.610293  9440 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.643555 10680 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:16.695178  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 273911us
I20260709 13:00:16.676417  9446 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:16.681550  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1839) took 1276 ms. Trace:
I20260709 13:00:16.683434  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1191) took 1957 ms. Trace:
I20260709 13:00:16.638269 10591 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:16.699489  8742 rpcz_store.cc:276] 0709 13:00:14.725644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.149371 (+423727us) service_pool.cc:224] Handling call
0709 13:00:15.149634 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.574268 (+424634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.669317 (+1095049us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54004669d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:16.683365 (+ 14048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946309376}
I20260709 13:00:16.701118  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.649869 10378 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.637723 10714 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.706298  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.720337  8740 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
W20260709 13:00:16.705447  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 284143us
I20260709 13:00:16.721606 10714 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:16.602710 10737 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.713173  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:16.630681  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.714926 10378 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.716230 10286 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.717070 10591 raft_consensus.cc:2764] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:16.717561  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:16.700497 10680 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.719465  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.677121 10454 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:16.726958  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.697741  8821 rpcz_store.cc:276] 0709 13:00:15.404775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.404898 (+   123us) service_pool.cc:224] Handling call
0709 13:00:15.405230 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.405584 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.405613 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.132035 (+726422us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b2c0005b728. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:00:16.132434 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.400863 (+268429us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:16.400902 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.400997 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.668159 (+267162us) raft_consensus.cc:1685] finished
0709 13:00:16.668181 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.681462 (+ 13281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10108,"raft.queue_time_us":331,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":14397,"spinlock_wait_cycles":29509376,"thread_start_us":268007,"threads_started":1,"wal-append.queue_time_us":370325}
I20260709 13:00:16.776283  8821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7884996e8b84433f80124e36111f58ac"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:16.777036  8821 raft_consensus.cc:606] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.777500  8821 raft_consensus.cc:614] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:16.646863 10554 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.792117  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.735493 10650 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:16.744836  8712 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.033s (SENT) [suppressed 17 similar messages]
W20260709 13:00:16.805043  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60274.6us
I20260709 13:00:16.765218  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:16.806406  8823 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:16.737319 10182 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.801901  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:16.766438 10680 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.646306  9443 rpcz_store.cc:276] 0709 13:00:14.907638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.909652 (+  2014us) service_pool.cc:224] Handling call
0709 13:00:16.388917 (+1479265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.413443 (+ 24526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.606202 (+192759us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400660dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:16.606373 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3325643520}
I20260709 13:00:16.808317 10182 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.809973  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:16.740645 10617 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.764690 10589 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.812883  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.823446 10182 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.811870  9441 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:16.791590  8745 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:16.803937 10557 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:16.815814  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 378004us
I20260709 13:00:16.809947 10666 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.803138 10733 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.815090 10600 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.858780  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.804250  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.865402  8823 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:16.810854 10521 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:16.819566 10591 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.876760  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:16.879151  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66388.4us
I20260709 13:00:16.791391 10454 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:16.882294  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.884359  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.889709  8821 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:16.896705  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:16.900756  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:16.903971  8894 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 6. Ops: []
I20260709 13:00:16.906204  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.907058  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.907452 10182 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.907897  8896 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:16.908150 10182 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.907080  8893 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:16.909700  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:16.910655  8896 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.912266  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:16.913223  8893 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:16.915653  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:16.916407  8898 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.833132 10617 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.919171  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.920277  8893 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:16.921636  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.852237 10589 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.922474  8823 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:16.920567  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.888739 10676 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.929527  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:16.924542  8894 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:16.924374  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.889129 10591 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.874754 10675 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:00:16.945387  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.898128 10521 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.946926  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.898104  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:16.959731  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.961639  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.947669  8895 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:16.877991 10666 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.956650  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:16.898394 10612 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:16.875130 10733 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:16.918793 10752 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.880539 10600 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:16.923580 10572 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.898850 10754 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.927455 10650 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.973026  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.927819 10676 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:16.979159  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:16.963950 10656 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.967078 10747 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:00:16.993431  8893 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:16.956399  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:16.999508  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.010003  8742 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:16.982750 10753 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.983606  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:17.004320  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.921823 10755 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:00:16.979888 10557 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.948923 10606 raft_consensus.cc:476] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:00:16.945441 10591 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.983430 10655 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:17.014307  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:17.058267  8818 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:17.001412  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:16.992854 10754 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.040315  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:17.061424  8742 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:17.043000  8820 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:17.059649  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:17.045238  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.005374 10676 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.991333 10572 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:16.978541 10661 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:00:16.597731  9444 rpcz_store.cc:276] 0709 13:00:15.189400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.209672 (+ 20272us) service_pool.cc:224] Handling call
0709 13:00:15.248020 (+ 38348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.257619 (+  9599us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:16.251315 (+993696us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b18003599e0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf
0709 13:00:16.251477 (+   162us) log.cc:844] Serialized 14 byte log entry
0709 13:00:16.497747 (+246270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.497806 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:16.497851 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.497916 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.505346 (+  7430us) inbound_call.cc:177] Queueing success response
0709 13:00:16.523642 (+ 18296us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b4800280158. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
Metrics: {"mutex_wait_us":36899,"prepare.queue_time_us":344874,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3337,"raft.queue_time_us":4990,"raft.run_cpu_time_us":1902,"raft.run_wall_time_us":23017,"spinlock_wait_cycles":20512768,"thread_start_us":558433,"threads_started":2,"wal-append.queue_time_us":415616}
I20260709 13:00:17.004722 10197 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:17.053908  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 97382.2us
I20260709 13:00:16.969313 10521 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:17.061502 10571 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.002s
I20260709 13:00:16.985456 10612 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.013695 10753 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:17.046286 10622 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:17.137809 10622 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.138343 10622 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.059621 10557 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.005591 10650 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.139719 10557 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.141695 10650 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:17.105259 10279 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 13:00:17.142629 10279 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.145215 10279 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.107950  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:17.080029 10554 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.115924 10378 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:17.148135 10632 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.614s	user 0.002s	sys 0.008s
W20260709 13:00:17.151894 10677 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.001s
I20260709 13:00:17.152426 10677 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.135859 10197 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.084179 10655 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:17.136663  9448 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:17.154368 10666 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.024739 10656 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:17.155678 10454 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:17.291419  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.294428  8743 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:17.067597  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:17.014189  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:17.302788  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.304152  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1847) took 1541 ms. Trace:
I20260709 13:00:17.111704  8896 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:17.390357 10662 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:17.391572  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:17.391360 10662 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:17.392604  8896 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:17.395191 10662 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.315802 10286 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.083515 10572 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.396960 10286 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:17.398010 10286 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.398136 10622 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.156875 10617 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:17.245726 10628 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.001s
I20260709 13:00:17.148710 10632 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.400990 10628 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.402100 10286 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.402666 10632 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:17.229278 10649 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.884s	user 0.015s	sys 0.000s
I20260709 13:00:17.403404 10628 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.255306 10676 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:17.403424  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:17.405979  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.406213 10676 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.247825 10677 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.251981 10680 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.408035 10510 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:17.408982 10510 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.409564 10680 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.157699 10753 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.411482 10510 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:17.066882 10754 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.425035  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:17.258051 10757 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:00:17.254105 10758 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:00:17.253861 10759 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:00:17.253552 10760 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:00:17.253449 10761 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:00:17.253260 10763 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:00:17.254174 10768 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:00:17.331043 10279 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:17.400342  8742 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:17.438920 10279 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.350517 10521 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:17.350917  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.353710 10589 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.357498 10656 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.456269  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.361783  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:17.458644  8891 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.355465  8815 rpcz_store.cc:276] 0709 13:00:15.762187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.769726 (+  7539us) service_pool.cc:224] Handling call
0709 13:00:16.554165 (+784439us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:16.554364 (+   199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.554734 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.554764 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.555202 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.859482 (+304280us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:16.859521 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.859586 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.303318 (+443732us) raft_consensus.cc:1685] finished
0709 13:00:17.303370 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.303850 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":18401024,"thread_start_us":295528,"threads_started":1,"wal-append.queue_time_us":428459}
I20260709 13:00:17.377614 10756 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:17.463376 10756 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:17.464287  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:17.379786  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.379989  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:17.340951  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.466153  8745 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:17.466403  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.381433 10636 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:17.385967 10197 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:17.386760 10571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40694 (request call id 11) took 7220 ms. Trace:
I20260709 13:00:17.395982 10662 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:17.398412 10591 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.235396 10587 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.423703  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.423837 10557 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:17.427410  8896 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 2: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:17.377055  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:17.407859  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:17.350301  8749 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.449608  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:17.451599 10572 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:17.258338 10655 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:17.462002  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 442) took 1009 ms. Trace:
I20260709 13:00:17.462004  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:17.467347  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.795341 10612 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.998576 10612 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.000173  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.001418  8746 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:18.001928  8898 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:18.003401 10591 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.002343 10571 rpcz_store.cc:276] 0709 13:00:10.169332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.199329 (+ 29997us) service_pool.cc:224] Handling call
0709 13:00:11.790176 (+1590847us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b4c002802c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f479ba20 00007f90f47aca29 00007f90f4772644 00007f90f4799f2d 00007f90f4799b93 00007f90f64281af 00007f90f6427f3e 00007f90f64ad888 00007f90f64ca75c 00007f90f64cb76a 00007f90f64ca91b
0709 13:00:11.791517 (+  1341us) log.cc:844] Serialized 506 byte log entry
0709 13:00:12.063087 (+271570us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd7a0a 00007f90f4dd7693 00007f90f4dd7631 00007f90f650bb52 00007f90f650bcf5 00007f90f6448691 00007f90f6428eaa 00007f90f642fd53 00007f90f642fcd9 00007f90f642fc69
0709 13:00:12.114424 (+ 51337us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:17.386443 (+5272019us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1161,"raft.run_cpu_time_us":1585,"raft.run_wall_time_us":195787,"spinlock_wait_cycles":68652032,"thread_start_us":74657,"threads_started":1,"wal-append.queue_time_us":137017,"wal-append.run_cpu_time_us":5503,"wal-append.run_wall_time_us":1221739}
I20260709 13:00:18.004802  8742 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:17.796518 10554 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.626701 10600 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.006923 10554 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:17.723498  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:17.675742 10766 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.587s	user 0.000s	sys 0.002s
I20260709 13:00:18.000335 10279 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.001780  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.018872 10632 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:18.027240  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1203) took 1301 ms. Trace:
I20260709 13:00:18.027623 10632 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:18.027705  8824 rpcz_store.cc:276] 0709 13:00:16.725185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.727548 (+  2363us) service_pool.cc:224] Handling call
0709 13:00:16.727956 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.728369 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.728404 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.728934 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.988680 (+259746us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.988730 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.988807 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.609133 (+620326us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.025765 (+416632us) raft_consensus.cc:1685] finished
0709 13:00:18.025786 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.026071 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2288,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5233,"spinlock_wait_cycles":32466688,"thread_start_us":259389,"threads_started":1,"wal-append.queue_time_us":359197}
I20260709 13:00:18.029112 10632 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.030395 10632 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.030707  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.031505  8824 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.999501  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:17.674242 10510 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:17.803823 10765 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.446s	user 0.001s	sys 0.000s
I20260709 13:00:17.842558 10564 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.230s
I20260709 13:00:17.845096 10521 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:17.803625  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:17.555229 10753 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.033259  8647 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 268) took 7887 ms. Trace:
I20260709 13:00:18.034868 10662 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:18.050557 10788 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.693s	user 0.002s	sys 0.000s
I20260709 13:00:18.051705 10662 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.052353 10559 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:18.035151  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 627888us
I20260709 13:00:18.009249 10622 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:17.538486 10703 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.035869  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 444) took 1569 ms. Trace:
I20260709 13:00:18.054890  9448 rpcz_store.cc:276] 0709 13:00:16.466028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.468265 (+  2237us) service_pool.cc:224] Handling call
0709 13:00:16.468639 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:17.373465 (+904826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.035344 (+661879us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b540050f854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:18.035620 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536305920}
W20260709 13:00:18.055745 10754 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.004s
I20260709 13:00:18.056207 10754 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.056583  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.056946  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1506) took 1057 ms. Trace:
I20260709 13:00:18.057382  8746 rpcz_store.cc:276] 0709 13:00:16.999598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.999773 (+   175us) service_pool.cc:224] Handling call
0709 13:00:18.056711 (+1056938us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54004641d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.056873 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159441152}
I20260709 13:00:18.058516 10754 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.055019 10182 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.059185  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.059648 10197 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.035687 10510 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.062480  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 278) took 1889 ms. Trace:
I20260709 13:00:18.062633  8892 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:18.063081 10510 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.062914  8816 rpcz_store.cc:276] 0709 13:00:16.173243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.174291 (+  1048us) service_pool.cc:224] Handling call
0709 13:00:16.361781 (+187490us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.062143 (+1700362us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.062311 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811961088}
I20260709 13:00:18.063926 10510 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.064509  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.064481 10510 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.037725  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:18.040700  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 438) took 1591 ms. Trace:
I20260709 13:00:18.042189 10572 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.039167 10737 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.066049  9441 rpcz_store.cc:276] 0709 13:00:16.448923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.453475 (+  4552us) service_pool.cc:224] Handling call
0709 13:00:16.570516 (+117041us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:16.570781 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.850423 (+279642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.040160 (+1189737us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54004a4554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:18.040525 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510824576}
I20260709 13:00:18.068084  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.065595  8816 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.068912  9441 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:18.044207  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 437) took 1596 ms. Trace:
I20260709 13:00:18.069624  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 451) took 1339 ms. Trace:
I20260709 13:00:18.070083  9441 rpcz_store.cc:276] 0709 13:00:16.729985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.067689 (+1337704us) service_pool.cc:224] Handling call
0709 13:00:18.069569 (+  1880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.071900  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.045959  8743 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:18.074838  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:18.065939 10197 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.077391  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.078518  9440 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:18.023581 10752 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.091900  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.025077  9442 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:18.092537  9446 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:18.092617  9440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 498) took 1583 ms (client timeout 2107 ms). Trace:
W20260709 13:00:18.093129  9440 rpcz_store.cc:269] 0709 13:00:16.509366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.076910 (+1567544us) service_pool.cc:224] Handling call
0709 13:00:18.092534 (+ 15624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.094333  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.096252  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:18.025583 10519 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:17.794585  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:17.545967 10425 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.043838  8749 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:18.104239  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1516) took 1062 ms. Trace:
I20260709 13:00:18.103906 10425 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.104578  8749 rpcz_store.cc:276] 0709 13:00:17.041766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.065202 (+ 23436us) service_pool.cc:224] Handling call
0709 13:00:18.104164 (+1038962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.105228  9448 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:18.105384  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1201) took 1695 ms. Trace:
I20260709 13:00:18.105918  8820 rpcz_store.cc:276] 0709 13:00:16.409524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.409683 (+   159us) service_pool.cc:224] Handling call
0709 13:00:17.042629 (+632946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.062081 (+ 19452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.104994 (+1042913us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54003fd9d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.105189 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455435008}
I20260709 13:00:18.107039  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.107707  8820 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:18.108192  8824 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:18.108382 10425 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.049626  8647 rpcz_store.cc:276] 0709 13:00:10.145761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.146013 (+   252us) service_pool.cc:224] Handling call
0709 13:00:10.431403 (+285390us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:00:18.032971 (+7601568us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:15.057965 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:15.148710 (+ 90745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:15.151620 (+  2910us) write_op.cc:435] Acquired schema lock
0709 13:00:15.151640 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:15.161742 (+ 10102us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:15.161826 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:15.161857 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:00:15.162287 (+   430us) tablet.cc:701] Row locks acquired
0709 13:00:15.162307 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:00:15.162457 (+   150us) write_op.cc:270] Start()
0709 13:00:15.162598 (+   141us) write_op.cc:276] Timestamp: P: 1783602015162409 usec, L: 0
0709 13:00:15.162631 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:00:15.163311 (+   680us) log.cc:844] Serialized 1687 byte log entry
0709 13:00:15.560292 (+396981us) op_driver.cc:464] REPLICATION: finished
0709 13:00:15.657800 (+ 97508us) write_op.cc:301] APPLY: starting
0709 13:00:15.657882 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:15.658035 (+   153us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:15.658156 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:15.667001 (+  8845us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:15.667161 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:00:15.667205 (+    44us) write_op.cc:312] APPLY: finished
0709 13:00:15.667414 (+   209us) log.cc:844] Serialized 41 byte log entry
0709 13:00:15.668833 (+  1419us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:15.668922 (+    89us) write_op.cc:454] Released schema lock
0709 13:00:15.668965 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:15.669271 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:15.669321 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":224818,"reported_tablets":9,"spinlock_wait_cycles":296362880,"child_traces":[["op",{"apply.queue_time_us":97259,"apply.run_cpu_time_us":2067,"apply.run_wall_time_us":11459,"num_ops":4,"prepare.queue_time_us":168685,"prepare.run_cpu_time_us":17178,"prepare.run_wall_time_us":216518,"raft.queue_time_us":90076,"raft.run_cpu_time_us":12109,"raft.run_wall_time_us":48874,"replication_time_us":397491,"thread_start_us":308793,"threads_started":4,"wal-append.queue_time_us":191946,"wal-append.run_cpu_time_us":9498,"wal-append.run_wall_time_us":1308946}]]}
W20260709 13:00:18.107048  9448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 449) took 1626 ms (client timeout 2041 ms). Trace:
I20260709 13:00:18.098791  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.109995  8744 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
W20260709 13:00:18.109338  9448 rpcz_store.cc:269] 0709 13:00:16.480584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.056225 (+1575641us) service_pool.cc:224] Handling call
0709 13:00:18.106882 (+ 50657us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54006dd654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.106974 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110331776}
I20260709 13:00:18.073884 10507 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:18.053138 10559 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:18.111343  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.111694 10507 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.112218 10559 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.069747  9447 rpcz_store.cc:276] 0709 13:00:16.447392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.448546 (+  1154us) service_pool.cc:224] Handling call
0709 13:00:16.467663 (+ 19117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.709224 (+241561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.709267 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.709828 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.896031 (+186203us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.896067 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.896139 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.600014 (+703875us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.043740 (+443726us) raft_consensus.cc:1685] finished
0709 13:00:18.043763 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.044035 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10947,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":27886,"spinlock_wait_cycles":15067520,"thread_start_us":419764,"threads_started":2,"wal-append.queue_time_us":321608}
W20260709 13:00:17.999660  8891 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.538s	user 0.015s	sys 0.000s
I20260709 13:00:18.113873 10507 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.046507  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.114121  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.048992 10753 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.103484 10571 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:18.115643 10571 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:18.116744  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 322436us
I20260709 13:00:18.116703 10571 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:18.111177  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.118072 10571 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.118382  8749 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:18.117228  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.119228 10564 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:18.119414  8816 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.081740  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.120960 10572 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.121388 10572 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:18.121614  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.121731 10572 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.121953  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.122795  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.123210  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.123131  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.124358  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.097676 10519 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.125458 10636 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.126611 10519 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.126927 10636 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.126979  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.127707 10636 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.128067  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1478) took 1215 ms. Trace:
I20260709 13:00:18.102087 10658 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:18.047205 10557 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.129036 10658 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:18.130010  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.130851  8896 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:18.130920 10785 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.757s	user 0.000s	sys 0.001s
I20260709 13:00:18.130640 10658 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.131876  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.132622  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1510) took 1130 ms. Trace:
I20260709 13:00:18.073877  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:18.082799  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.045377 10521 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.037461  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.084703  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.085950  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.087000  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:17.999568  9449 rpcz_store.cc:276] 0709 13:00:16.452039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.452199 (+   160us) service_pool.cc:224] Handling call
0709 13:00:16.467164 (+ 14965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.502172 (+ 35008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.502214 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.507726 (+  5512us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.891430 (+383704us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:17.054504 (+163074us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.054726 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.459152 (+404426us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:17.459238 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.459339 (+   101us) raft_consensus.cc:1685] finished
0709 13:00:17.461599 (+  2260us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.461834 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39924,"prepare.queue_time_us":197440,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":29,"raft.queue_time_us":48317,"raft.run_cpu_time_us":509,"raft.run_wall_time_us":78644,"spinlock_wait_cycles":81320960,"thread_start_us":406448,"threads_started":2,"wal-append.queue_time_us":518178}
I20260709 13:00:17.999483 10612 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.019745  8742 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:18.034735  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.060642  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.088639  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.081436  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.090940 10783 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.672s	user 0.000s	sys 0.002s
I20260709 13:00:18.091609 10752 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.024335  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.090804 10554 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.113828  8891 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:18.073084  9441 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:18.114981  9445 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:18.110545  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1519) took 1067 ms. Trace:
W20260709 13:00:18.120355  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45882.9us
I20260709 13:00:18.087713 10756 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.133041  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.119738 10564 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:18.120560  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.186450  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.141438  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.111357  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.103181 10510 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.128502  8889 rpcz_store.cc:276] 0709 13:00:16.912182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.918244 (+  6062us) service_pool.cc:224] Handling call
0709 13:00:18.126686 (+1208442us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.127998 (+  1312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14756736}
I20260709 13:00:18.180289  8744 rpcz_store.cc:276] 0709 13:00:17.042654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.098312 (+1055658us) service_pool.cc:224] Handling call
0709 13:00:18.110478 (+ 12166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.140863 10182 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.193295  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1517) took 1151 ms. Trace:
I20260709 13:00:18.128509  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:18.103343  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.193851  8749 rpcz_store.cc:276] 0709 13:00:17.042082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.120295 (+1078213us) service_pool.cc:224] Handling call
0709 13:00:18.193217 (+ 72922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.196189  9448 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:18.221928  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:18.101003 10767 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.041s	user 0.005s	sys 0.015s
I20260709 13:00:18.224191  9447 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:18.226740  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.228343  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.134234 10636 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.134353  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:18.223444  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 448) took 1733 ms. Trace:
I20260709 13:00:18.234835  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.235055  9448 rpcz_store.cc:276] 0709 13:00:16.471911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.111056 (+1639145us) service_pool.cc:224] Handling call
0709 13:00:18.205661 (+ 94605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.235587  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.235625  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.221472  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.140103 10519 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.193094 10658 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:18.241602  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 113315us
I20260709 13:00:18.243021  9446 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:00:18.243548  9446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 504) took 1629 ms (client timeout 1619 ms). Trace:
W20260709 13:00:18.245513  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 172044us
I20260709 13:00:18.245944  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:18.243999  9446 rpcz_store.cc:269] 0709 13:00:16.613922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.022982 (+1409060us) service_pool.cc:224] Handling call
0709 13:00:18.243487 (+220505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.248126  8749 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:18.233649 10182 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.248864  8894 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 1.
I20260709 13:00:18.133531 10735 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:18.234943  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 440) took 1775 ms. Trace:
W20260709 13:00:18.220940  9441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 495) took 1695 ms (client timeout 2086 ms). Trace:
I20260709 13:00:18.133015  8747 rpcz_store.cc:276] 0709 13:00:17.002279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.002416 (+   137us) service_pool.cc:224] Handling call
0709 13:00:17.002750 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:17.289258 (+286508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.289381 (+   123us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:17.290737 (+  1356us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.301749 (+ 11012us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:17.301807 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.306140 (+  4333us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.130178 (+824038us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.132211 (+  2033us) raft_consensus.cc:1685] finished
0709 13:00:18.132231 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.132452 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61656,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8609,"spinlock_wait_cycles":72851712,"thread_start_us":295889,"threads_started":2,"wal-append.queue_time_us":81921}
I20260709 13:00:18.255013  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.255846  8824 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.192874 10572 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:18.250972  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 436280us
I20260709 13:00:18.221810  9444 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
W20260709 13:00:18.243448  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 449) took 1760 ms (client timeout 1814 ms). Trace:
I20260709 13:00:18.262113  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.251997  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.259021  8822 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.267848  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.257278  8817 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.268780  9448 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:18.269763  8740 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.262008  8746 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:18.271548  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:18.272156 10519 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.274626 10519 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.275228 10554 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:18.276146  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.276433  9449 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:18.263301  9443 rpcz_store.cc:269] 0709 13:00:16.472214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.084369 (+1612155us) service_pool.cc:224] Handling call
0709 13:00:18.232863 (+148494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.282024  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1523) took 1238 ms. Trace:
I20260709 13:00:18.282670  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.284818  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.260545 10425 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:18.270589  8742 pending_rounds.cc:85] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 1
I20260709 13:00:18.275784 10554 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:18.263414 10753 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.291002 10182 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.263792  8745 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:18.282472  8747 rpcz_store.cc:276] 0709 13:00:17.043858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.280784 (+1236926us) service_pool.cc:224] Handling call
0709 13:00:18.281169 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.281650 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.281676 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.281739 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:18.281767 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.281824 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.281939 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.253311  9441 rpcz_store.cc:269] 0709 13:00:16.499618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.071381 (+1571763us) service_pool.cc:224] Handling call
0709 13:00:18.194788 (+123407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.291587  8744 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:18.287850  9447 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:18.266036  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.264183  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 503) took 1664 ms (client timeout 1768 ms). Trace:
I20260709 13:00:18.276746  8894 raft_consensus.cc:2376] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3. Re-sending same reply.
I20260709 13:00:18.251144  9445 rpcz_store.cc:276] 0709 13:00:16.450513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.455228 (+  4715us) service_pool.cc:224] Handling call
0709 13:00:18.046154 (+1590926us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.220782 (+174628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8276224}
I20260709 13:00:18.336483  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 502) took 1762 ms. Trace:
I20260709 13:00:18.340163  9447 rpcz_store.cc:276] 0709 13:00:16.574572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.284530 (+1709958us) service_pool.cc:224] Handling call
0709 13:00:18.336406 (+ 51876us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.291559  8742 pending_rounds.cc:107] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:18.346061  8742 raft_consensus.cc:2889] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:18.343665  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:18.282425 10778 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.023s	user 0.002s	sys 0.000s
I20260709 13:00:18.371416  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:18.285766  8891 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:18.291265 10753 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.395965  8747 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:18.414296  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.267939 10735 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.425318  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.432374  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.437278  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1199) took 1120 ms. Trace:
I20260709 13:00:18.437830  8747 rpcz_store.cc:276] 0709 13:00:17.291899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.324131 (+1032232us) service_pool.cc:224] Handling call
0709 13:00:18.395750 (+ 71619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.412298 (+ 16548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.412559 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.341807 10425 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:18.462826  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:18.463670  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 427472us
I20260709 13:00:18.463788  9441 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:00:18.468842  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 182608us
I20260709 13:00:18.403424  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1208) took 1018 ms. Trace:
I20260709 13:00:18.475606  8745 rpcz_store.cc:276] 0709 13:00:17.377917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.034319 (+656402us) service_pool.cc:224] Handling call
0709 13:00:18.395985 (+361666us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.476835 10554 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.477725 10437 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:18.478034 10754 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.478494 10437 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:18.478667 10754 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.480082  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:18.479815 10437 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.481269 10662 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.481666 10662 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.482019 10662 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.476199  8822 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:18.409543 10753 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.512635  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1853) took 1568 ms. Trace:
I20260709 13:00:18.513419  8823 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:00:18.513895  8823 pending_rounds.cc:85] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 3
I20260709 13:00:18.514379  8823 pending_rounds.cc:107] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:00:18.342123  9444 rpcz_store.cc:269] 0709 13:00:16.599932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.061133 (+1461201us) service_pool.cc:224] Handling call
0709 13:00:18.262551 (+201418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.516201  8823 raft_consensus.cc:2889] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:18.514751  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.513142  8821 rpcz_store.cc:276] 0709 13:00:16.943669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.944572 (+   903us) service_pool.cc:224] Handling call
0709 13:00:16.944864 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.957994 (+ 13130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.958044 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.958641 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.302793 (+344152us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:17.303428 (+   635us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.303859 (+   431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.304040 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.281799 (+977759us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.512128 (+230329us) raft_consensus.cc:1685] finished
0709 13:00:18.512244 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.512440 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12709,"prepare.queue_time_us":28077,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":2374,"raft.queue_time_us":36848,"raft.run_cpu_time_us":596,"raft.run_wall_time_us":7973,"spinlock_wait_cycles":320776576,"thread_start_us":293881,"threads_started":1,"wal-append.queue_time_us":270323}
I20260709 13:00:18.519769  9445 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:18.520903  9440 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:00:18.522372 10784 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.141s	user 0.001s	sys 0.002s
I20260709 13:00:18.522524 10735 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.410672 10559 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:18.520736  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.526475  9447 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:18.524747  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1518) took 1482 ms. Trace:
I20260709 13:00:18.465075  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.532073  8744 rpcz_store.cc:276] 0709 13:00:17.042368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.254267 (+1211899us) service_pool.cc:224] Handling call
0709 13:00:18.288539 (+ 34272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.319762 (+ 31223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.524682 (+204920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.535022  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.536660  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.536526  9448 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:18.539026  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 277842us
I20260709 13:00:18.541016  8740 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:18.541628  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1200) took 1249 ms. Trace:
I20260709 13:00:18.541781  8820 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:18.541963  8740 rpcz_store.cc:276] 0709 13:00:17.292430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.227762 (+935332us) service_pool.cc:224] Handling call
0709 13:00:18.541568 (+313806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.544377  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.546020  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.546509  8898 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.547704  8740 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:18.548972  8824 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:18.527164 10521 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.549664  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1207) took 1170 ms. Trace:
I20260709 13:00:18.550122  8740 rpcz_store.cc:276] 0709 13:00:17.377566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.545087 (+1167521us) service_pool.cc:224] Handling call
0709 13:00:18.545343 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.548131 (+  2788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.548267 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.344530 10564 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:18.542296  8745 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:18.555255  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.552297 10612 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:18.548390 10569 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.548624 10591 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.544018  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.557045 10564 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.562608  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.565398  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 448) took 2084 ms. Trace:
I20260709 13:00:18.565866  9447 rpcz_store.cc:276] 0709 13:00:16.477908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.536437 (+2058529us) service_pool.cc:224] Handling call
0709 13:00:18.558363 (+ 21926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.562126 (+  3763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.562144 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.562174 (+    30us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:18.562192 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.562228 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.562298 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.566560  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.557238 10569 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.571045  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.571627  8821 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.570053 10735 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:18.574811 10735 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:18.575682  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:18.575140 10735 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:18.579041  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.559283 10425 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.579486  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 447) took 2104 ms. Trace:
I20260709 13:00:18.577277  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.559810 10559 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.569392  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1205) took 1192 ms. Trace:
I20260709 13:00:18.575157 10569 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.601871 10739 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.599424  9445 rpcz_store.cc:276] 0709 13:00:16.475120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.520614 (+2045494us) service_pool.cc:224] Handling call
0709 13:00:18.551516 (+ 30902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.568160 (+ 16644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.570489 (+  2329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.572098 (+  1609us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:18.578047 (+  5949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.578121 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.579430 (+  1309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.602648  8820 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:00:18.603753  8824 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:18.578388  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:18.605845  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.599979  8891 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:18.607591  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1212) took 1225 ms. Trace:
I20260709 13:00:18.580190  9444 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:18.609560  8740 rpcz_store.cc:276] 0709 13:00:17.379449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.551500 (+1172051us) service_pool.cc:224] Handling call
0709 13:00:18.572901 (+ 21401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.604453 (+ 31552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.604485 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.604514 (+    29us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:18.604533 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.604574 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.604648 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.582569  8743 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.001s
I20260709 13:00:18.617785  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1811) took 1208 ms. Trace:
I20260709 13:00:18.621443 10182 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:18.621891 10182 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:18.622171  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.622229 10182 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:18.621376  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53781.5us
W20260709 13:00:18.622670  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.626107  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.626852  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1487) took 1622 ms. Trace:
I20260709 13:00:18.627919  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.628423  9447 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:18.603060 10799 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
W20260709 13:00:18.629848  9442 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:00:18.630040 10799 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:18.630225  9442 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:18.630549 10799 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:18.630856  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 446) took 2156 ms (client timeout 1709 ms). Trace:
I20260709 13:00:18.631654 10799 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.632936 10799 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.635720  8889 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:18.636684 10739 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.637684  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 290) took 1169 ms. Trace:
I20260709 13:00:18.638067  8815 rpcz_store.cc:276] 0709 13:00:17.468487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.472224 (+  3737us) service_pool.cc:224] Handling call
0709 13:00:17.472847 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:17.587362 (+114515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.587402 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:17.645745 (+ 58343us) log.cc:844] Serialized 26 byte log entry
0709 13:00:18.262970 (+617225us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:18.267255 (+  4285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.276785 (+  9530us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.637460 (+360675us) raft_consensus.cc:1685] finished
0709 13:00:18.637488 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.637610 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114037,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":137,"thread_start_us":236796,"threads_started":1,"wal-append.queue_time_us":818229}
I20260709 13:00:18.639914 10739 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:18.627341  8895 rpcz_store.cc:276] 0709 13:00:16.916091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.963298 (+ 47207us) service_pool.cc:224] Handling call
0709 13:00:16.992975 (+ 29677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:17.039970 (+ 46995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.039996 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:17.055923 (+ 15927us) log.cc:844] Serialized 26 byte log entry
0709 13:00:18.043062 (+987139us) spinlock_profiling.cc:243] Waited 990 us on lock 0x7b18002f9e60. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:00:18.292513 (+249451us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:18.341914 (+ 49401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.500711 (+158797us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.516838 (+ 16127us) raft_consensus.cc:1685] finished
0709 13:00:18.516861 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.538113 (+ 21252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46602,"raft.queue_time_us":46777,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":35049,"spinlock_wait_cycles":2277120,"thread_start_us":216740,"threads_started":1,"wal-append.queue_time_us":1380108}
I20260709 13:00:18.606681  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.581502 10591 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.619362 10754 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:18.631343  9442 rpcz_store.cc:269] 0709 13:00:16.474599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.448941 (+974342us) service_pool.cc:224] Handling call
0709 13:00:18.630796 (+1181855us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.649026  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.650240  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.649694 10735 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.621045  8743 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:18.666065  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1218) took 1275 ms. Trace:
I20260709 13:00:18.662236  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.626025  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.605599  8745 rpcz_store.cc:276] 0709 13:00:17.376647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.476678 (+1100031us) service_pool.cc:224] Handling call
0709 13:00:18.477107 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.557872 (+ 80765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.568215 (+ 10343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.702781  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:18.620787  8891 rpcz_store.cc:276] 0709 13:00:17.409317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.492940 (+1083623us) service_pool.cc:224] Handling call
0709 13:00:18.617699 (+124759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.700974  8895 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:18.710390  8740 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:18.712770  9447 raft_consensus.cc:2376] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5. Re-sending same reply.
I20260709 13:00:18.720877  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1214) took 1340 ms. Trace:
I20260709 13:00:18.721593  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 323) took 1309 ms. Trace:
I20260709 13:00:18.721426  8740 rpcz_store.cc:276] 0709 13:00:17.380104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.613625 (+1233521us) service_pool.cc:224] Handling call
0709 13:00:18.619246 (+  5621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.720527 (+101281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.720797 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.722441  8895 rpcz_store.cc:276] 0709 13:00:17.412215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.661920 (+1249705us) service_pool.cc:224] Handling call
0709 13:00:18.721518 (+ 59598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.666441  8743 rpcz_store.cc:276] 0709 13:00:17.390464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.657449 (+266985us) service_pool.cc:224] Handling call
0709 13:00:18.666000 (+1008551us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.723856  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.645363  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 499) took 2083 ms. Trace:
I20260709 13:00:18.695631  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 294) took 1148 ms. Trace:
I20260709 13:00:18.736495  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.738816  9441 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:18.739497  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 452) took 2091 ms. Trace:
I20260709 13:00:18.739809  9441 rpcz_store.cc:276] 0709 13:00:16.647623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.738221 (+2090598us) service_pool.cc:224] Handling call
0709 13:00:18.738601 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.739243 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.739431 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.733094  9445 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.742503 10591 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.729727  9444 rpcz_store.cc:276] 0709 13:00:16.557809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.533894 (+1976085us) service_pool.cc:224] Handling call
0709 13:00:18.554258 (+ 20364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.613275 (+ 59017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.641008 (+ 27733us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.743412 10733 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.744715  9444 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:18.735369 10735 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.745519  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 455) took 1306 ms. Trace:
W20260709 13:00:18.746205  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 500) took 2185 ms (client timeout 2817 ms). Trace:
I20260709 13:00:18.746172  9444 rpcz_store.cc:276] 0709 13:00:17.438882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.743904 (+1305022us) service_pool.cc:224] Handling call
0709 13:00:18.744299 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.745190 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.745450 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.746766  9449 rpcz_store.cc:269] 0709 13:00:16.561234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.180739 (+1619505us) service_pool.cc:224] Handling call
0709 13:00:18.746029 (+565290us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.746155 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87843584}
I20260709 13:00:18.756356  8898 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.757217  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 321) took 1345 ms. Trace:
I20260709 13:00:18.720278  8817 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.767666  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:18.751842  8891 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 2: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:18.745411  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.771733  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 450) took 2144 ms. Trace:
I20260709 13:00:18.757730  8898 rpcz_store.cc:276] 0709 13:00:17.411410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.524808 (+1113398us) service_pool.cc:224] Handling call
0709 13:00:18.757142 (+232334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.780790 10662 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.783293 10662 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.786162  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.730751  8816 rpcz_store.cc:276] 0709 13:00:17.546973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.116980 (+570007us) service_pool.cc:224] Handling call
0709 13:00:18.644986 (+528006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.774451  9448 rpcz_store.cc:276] 0709 13:00:16.639758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.548159 (+1908401us) service_pool.cc:224] Handling call
0709 13:00:18.564308 (+ 16149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:18.572316 (+  8008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.572349 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:18.582249 (+  9900us) log.cc:844] Serialized 26 byte log entry
0709 13:00:18.723929 (+141680us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:18.723968 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.724032 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.765929 (+ 41897us) raft_consensus.cc:1685] finished
0709 13:00:18.765962 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.766153 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2981,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3132,"thread_start_us":102820,"threads_started":1,"wal-append.queue_time_us":156209}
I20260709 13:00:18.791546  9447 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:18.793372  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1817) took 1382 ms. Trace:
I20260709 13:00:18.795382  8895 rpcz_store.cc:276] 0709 13:00:17.411108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.751265 (+1340157us) service_pool.cc:224] Handling call
0709 13:00:18.793309 (+ 42044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:18.781075  9440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 447) took 2309 ms (client timeout 2177 ms). Trace:
I20260709 13:00:18.794019  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:18.799595  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.800797  9447 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:18.796195  9440 rpcz_store.cc:269] 0709 13:00:16.471663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.093951 (+1622288us) service_pool.cc:224] Handling call
0709 13:00:18.780863 (+686912us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.781001 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924832000}
I20260709 13:00:18.805245 10182 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.806275  8898 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:18.806537  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.796402  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.816918 10808 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:00:18.816985 10737 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.817770 10737 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:18.818553 10771 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.557s	user 0.005s	sys 0.007s
I20260709 13:00:18.820068 10737 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.820343 10753 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.821534  9443 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:18.810761 10559 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.822999  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 451) took 2176 ms. Trace:
I20260709 13:00:18.823149  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.823701  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.824162  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.824474  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.824625  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.826731  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.820076 10538 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.834875 10182 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.836124  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.842638  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 2.
I20260709 13:00:18.810628 10662 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:18.844077  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 318) took 1433 ms. Trace:
I20260709 13:00:18.823400  9443 rpcz_store.cc:276] 0709 13:00:16.646339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.282335 (+1635996us) service_pool.cc:224] Handling call
0709 13:00:18.822787 (+540452us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54003a2854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:18.822939 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237854464}
I20260709 13:00:18.842404  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.854038  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.952977  8896 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
W20260709 13:00:18.856563  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:18.859076  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 55712.1us
I20260709 13:00:18.841799 10538 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:18.849869  8894 raft_consensus.cc:2368] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:18.843602  8788 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:18.846447  8895 rpcz_store.cc:276] 0709 13:00:17.410452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.801930 (+1391478us) service_pool.cc:224] Handling call
0709 13:00:18.844020 (+ 42090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.957515  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 313) took 1548 ms. Trace:
I20260709 13:00:18.845818 10662 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:18.959017 10662 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.956543  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:18.962193 10781 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.543s	user 0.006s	sys 0.011s
I20260709 13:00:18.963124  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:18.957896  8894 rpcz_store.cc:276] 0709 13:00:17.408852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.348015 (+939163us) service_pool.cc:224] Handling call
0709 13:00:18.826637 (+478622us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.957387 (+130750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4716800}
I20260709 13:00:18.965394  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:18.955643 10571 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.837s	user 0.012s	sys 0.012s
I20260709 13:00:18.968240  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.968708  8896 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:18.969674  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 127322us
I20260709 13:00:18.970168  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:18.971967  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.976649  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1488) took 2050 ms. Trace:
I20260709 13:00:18.976902  8897 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:18.978010  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1512) took 1975 ms. Trace:
I20260709 13:00:18.978986 10538 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.976708 10605 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.983036  8892 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:18.983556  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1812) took 1574 ms. Trace:
W20260709 13:00:18.983856  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 1.52837e+06us
I20260709 13:00:18.984211  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1489) took 1634 ms. Trace:
I20260709 13:00:18.966931 10591 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:18.985071  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 157824us
W20260709 13:00:18.984567  8713 connection.cc:256] RPC call timeout handler was delayed by -1.4912515190000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:00:18.985831  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 941533us
I20260709 13:00:18.986788  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:18.987108 10753 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:18.988591 10424 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:18.974738  8889 raft_consensus.cc:2410] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:18.989413  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.987881  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.989588  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1511) took 1972 ms. Trace:
W20260709 13:00:18.990231  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.985572 10591 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:18.990780  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.989217  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:18.991101  8712 connection.cc:256] RPC call timeout handler was delayed by -1.0037177299999485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 13:00:18.977113  8893 rpcz_store.cc:276] 0709 13:00:16.916380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.008426 (+ 92046us) service_pool.cc:224] Handling call
0709 13:00:17.056325 (+ 47899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:17.056627 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.056653 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:17.057028 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.380718 (+323690us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:17.381214 (+   496us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.383513 (+  2299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.383660 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.596577 (+212917us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b48001d01d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:00:18.961548 (+1364971us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.965055 (+  3507us) raft_consensus.cc:1685] finished
0709 13:00:18.965073 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.967052 (+  1979us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269,"prepare.queue_time_us":1781,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":11534,"raft.run_cpu_time_us":3416,"raft.run_wall_time_us":518843,"spinlock_wait_cycles":35245568,"thread_start_us":281190,"threads_started":1,"wal-append.queue_time_us":336333}
I20260709 13:00:18.992172  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.992755  8893 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:18.992784  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:18.993034  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:18.993443 10557 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.978385  8741 rpcz_store.cc:276] 0709 13:00:17.002982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.003123 (+   141us) service_pool.cc:224] Handling call
0709 13:00:17.003458 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.003635 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:17.293337 (+289702us) log.cc:844] Serialized 14 byte log entry
0709 13:00:18.977517 (+1684180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.977601 (+    84us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:18.977643 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.977737 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.977858 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1634740,"prepare.queue_time_us":257894,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"spinlock_wait_cycles":64893312,"thread_start_us":334852,"threads_started":2,"wal-append.queue_time_us":90171}
I20260709 13:00:18.994271  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:18.994488  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:18.994514  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:18.995160  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:18.995729  8745 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:18.996577 10632 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:18.997354 10632 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.997504 10753 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:18.997781 10632 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.998062 10753 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:18.984460 10662 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.998240 10632 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.999106 10753 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:18.999965 10662 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:18.998065  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:19.001040  8889 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.984725  8890 rpcz_store.cc:276] 0709 13:00:17.352432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.369227 (+ 16795us) service_pool.cc:224] Handling call
0709 13:00:17.404635 (+ 35408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.464931 (+ 60296us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:17.477015 (+ 12084us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.827237 (+350222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:17.832829 (+  5592us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:17.866963 (+ 34134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.851614 (+984651us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b18002f9d40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419
0709 13:00:18.958595 (+106981us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b54003bd154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:18.965571 (+  6976us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.970535 (+  4964us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81401,"prepare.queue_time_us":281,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":4667,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":18197,"spinlock_wait_cycles":142486016,"thread_start_us":255080,"threads_started":1,"wal-append.queue_time_us":564152}
W20260709 13:00:19.003353  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.004098  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1515) took 1988 ms. Trace:
I20260709 13:00:19.004598 10774 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.003270  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1521) took 1960 ms. Trace:
I20260709 13:00:18.994886 10655 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:18.983851  8892 rpcz_store.cc:276] 0709 13:00:17.409599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.229643 (+820044us) service_pool.cc:224] Handling call
0709 13:00:18.971865 (+742222us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.983505 (+ 11640us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8818560}
I20260709 13:00:19.004146  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1206) took 1627 ms. Trace:
I20260709 13:00:18.997169  9412 leader_election.cc:400] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:19.004549  8749 rpcz_store.cc:276] 0709 13:00:17.016200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.248744 (+1232544us) service_pool.cc:224] Handling call
0709 13:00:18.278734 (+ 29990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.279014 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:18.451174 (+172160us) log.cc:844] Serialized 14 byte log entry
0709 13:00:19.003813 (+552639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.003867 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:19.003900 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.003956 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.004046 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309573,"prepare.queue_time_us":315867,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":27,"thread_start_us":316810,"threads_started":2,"wal-append.queue_time_us":247465}
I20260709 13:00:18.994040 10557 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.014716  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.015220  8895 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:19.005914  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:18.987169  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:19.007337  8898 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:18.990566  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.012116  8896 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:19.016117  8893 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.006229  8746 rpcz_store.cc:276] 0709 13:00:17.043278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.271550 (+1228272us) service_pool.cc:224] Handling call
0709 13:00:18.272045 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.272335 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:18.459846 (+187511us) log.cc:844] Serialized 14 byte log entry
0709 13:00:19.002939 (+543093us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.002991 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:19.003029 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.003090 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.003200 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":392525,"prepare.queue_time_us":292644,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":248094,"threads_started":2,"wal-append.queue_time_us":104376}
I20260709 13:00:19.013015  8744 rpcz_store.cc:276] 0709 13:00:17.377086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.534669 (+1157583us) service_pool.cc:224] Handling call
0709 13:00:18.546689 (+ 12020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.604755 (+ 58066us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:18.644136 (+ 39381us) log.cc:844] Serialized 14 byte log entry
0709 13:00:19.003894 (+359758us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.003936 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:19.003967 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.004019 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.004104 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":233074,"prepare.queue_time_us":31829,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3483,"thread_start_us":28907,"threads_started":1,"wal-append.queue_time_us":75559}
I20260709 13:00:19.015900  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:19.016435  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46338.5us
I20260709 13:00:19.025146  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.026007  8896 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:19.026815  8895 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:18.967820  8891 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:18.990787 10591 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.027247  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:19.003428  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:18.990065  8748 rpcz_store.cc:276] 0709 13:00:17.002651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.003378 (+   727us) service_pool.cc:224] Handling call
0709 13:00:17.313312 (+309934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.313937 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:17.314536 (+   599us) log.cc:844] Serialized 14 byte log entry
0709 13:00:18.049820 (+735284us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:18.958122 (+908302us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b180045b2c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:00:18.971176 (+ 13054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.971233 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:18.971258 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.971297 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.974943 (+  3646us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5536,"prepare.queue_time_us":17387,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":1646232,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"spinlock_wait_cycles":26568832,"thread_start_us":15656,"threads_started":1,"wal-append.queue_time_us":30533}
I20260709 13:00:19.022881 10437 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.036492  8894 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:19.036721  8892 raft_consensus.cc:2368] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:19.041112  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.043322  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.023944 10605 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:19.045240  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48288.3us
I20260709 13:00:19.047668  9412 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.049180 10661 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.049855 10661 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.052335 10661 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:19.052927  8712 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.504s (SENT)
I20260709 13:00:19.057915  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.058625 10557 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.059437  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.060593  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.061618  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.062646  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.063860  8819 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:18.984211 10649 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:00:19.065173  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.065570  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.065198  8819 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:19.067006  8821 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:19.068101  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.027956 10774 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.069887  8824 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:19.071456 10774 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.074813  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.076058  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.076162 10632 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.079137  8889 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:19.081090 10632 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.081264  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:19.081758  8893 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:19.030627 10662 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:19.077060  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.053937  8822 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:19.054749 10815 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:00:19.054847  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.086786  8748 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.055133  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:19.055457 10813 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:00:19.055833 10587 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.042543 10437 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.056526  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.057400  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1824) took 1058 ms. Trace:
I20260709 13:00:19.119746  8898 rpcz_store.cc:276] 0709 13:00:17.999250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.808503 (+809253us) service_pool.cc:224] Handling call
0709 13:00:19.057205 (+248702us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003be054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:19.057346 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391739648}
I20260709 13:00:19.073140  8740 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:19.057570  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.043679 10589 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:19.055146 10814 ts_tablet_manager.cc:933] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:19.091575  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55168us
W20260709 13:00:19.091745  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 48681us
I20260709 13:00:19.037374 10655 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.092485 10662 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:19.095232  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.127657  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1474) took 2221 ms (client timeout 1565 ms). Trace:
W20260709 13:00:19.055948 10786 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.672s	user 0.005s	sys 0.012s
I20260709 13:00:19.117969  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.129757  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:19.129683  8897 rpcz_store.cc:269] 0709 13:00:16.906143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.906280 (+   137us) service_pool.cc:224] Handling call
0709 13:00:18.047550 (+1141270us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:18.081067 (+ 33517us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aae9c 00007f90f33aaa5b 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:19.127450 (+1046383us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400094cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:19.127582 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904942592}
I20260709 13:00:19.130589  8746 raft_consensus.cc:2368] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:19.131327  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.131531  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.131637  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.132138  8892 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:19.120081  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:19.087783  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.133181  8894 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:19.132633  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1827) took 1132 ms. Trace:
I20260709 13:00:19.133841  8892 rpcz_store.cc:276] 0709 13:00:18.000208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.126764 (+1126556us) service_pool.cc:224] Handling call
0709 13:00:19.132582 (+  5818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.135612  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.136219  8897 raft_consensus.cc:2376] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:19.093415 10571 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:00:19.137451  8897 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:19.138033  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1803) took 1778 ms. Trace:
I20260709 13:00:19.138345  8897 rpcz_store.cc:276] 0709 13:00:17.365496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.136859 (+1771363us) service_pool.cc:224] Handling call
0709 13:00:19.137176 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.137795 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.137951 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.138988  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:19.139950  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1805) took 1739 ms. Trace:
I20260709 13:00:19.140321  8897 rpcz_store.cc:276] 0709 13:00:17.400792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.139124 (+1738332us) service_pool.cc:224] Handling call
0709 13:00:19.139415 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.139586 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.139619 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.139676 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:19.139721 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.139784 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.139881 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.141234  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1807) took 1733 ms. Trace:
I20260709 13:00:19.142270 10661 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:19.142935 10661 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:19.099249  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.120697  8740 pending_rounds.cc:85] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 5
I20260709 13:00:19.143710  8740 pending_rounds.cc:107] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:19.045449  8890 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:19.144057  8740 raft_consensus.cc:2889] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:19.123961  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:19.121579  8817 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:19.127362  8745 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:19.118588 10587 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.097316 10612 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.117244 10632 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.150589  8822 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:19.184194  8749 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.127614  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:19.163242  8894 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 3. Ops: [1.3-1.3]
I20260709 13:00:19.185554  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1801) took 1826 ms. Trace:
I20260709 13:00:19.185967  8894 rpcz_store.cc:276] 0709 13:00:17.359162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.134021 (+1774859us) service_pool.cc:224] Handling call
0709 13:00:19.134655 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.185139 (+ 50484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.185481 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.121680 10589 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.164178  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1858) took 2159 ms. Trace:
I20260709 13:00:19.190215  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.122061 10437 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.154102  8892 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.083616  8643 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 4, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.165436  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:19.132583  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:19.188905  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.194100  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.110673  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 453) took 1722 ms. Trace:
I20260709 13:00:19.189953  8823 rpcz_store.cc:276] 0709 13:00:16.995743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.996127 (+   384us) service_pool.cc:224] Handling call
0709 13:00:17.053383 (+ 57256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.537879 (+1484496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.117759 (+579880us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540012b954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:19.154974 (+ 37215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2834836864}
I20260709 13:00:19.197599  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 315) took 1788 ms. Trace:
I20260709 13:00:19.201435  8895 rpcz_store.cc:276] 0709 13:00:17.409477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.145838 (+1736361us) service_pool.cc:224] Handling call
0709 13:00:19.181684 (+ 35846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.197216 (+ 15532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.197242 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.197324 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:19.197367 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.197424 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.197537 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.202034  8897 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:00:19.202555  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.202682  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1809) took 1794 ms. Trace:
W20260709 13:00:19.198534  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71020.8us
I20260709 13:00:19.199033  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.203114  8897 rpcz_store.cc:276] 0709 13:00:17.408669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.146831 (+1738162us) service_pool.cc:224] Handling call
0709 13:00:19.183449 (+ 36618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.202459 (+ 19010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.202630 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:19.203495  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.141827  8889 rpcz_store.cc:276] 0709 13:00:17.407941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.140419 (+1732478us) service_pool.cc:224] Handling call
0709 13:00:19.140706 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.140911 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.140938 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.140978 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:19.141018 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.141074 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.141169 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:19.204334 10812 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:00:19.205628  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 322) took 1793 ms. Trace:
I20260709 13:00:19.180341 10587 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.208218  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1802) took 1843 ms. Trace:
I20260709 13:00:19.209163  8892 rpcz_store.cc:276] 0709 13:00:17.359642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.134568 (+1774926us) service_pool.cc:224] Handling call
0709 13:00:19.134856 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.202459 (+ 67603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.202630 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.210811  8892 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:19.097105 10605 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.183960 10632 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.191946 10589 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.206060  8889 rpcz_store.cc:276] 0709 13:00:17.411692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.204560 (+1792868us) service_pool.cc:224] Handling call
0709 13:00:19.204913 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.205184 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.205247 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.205330 (+    83us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:19.205376 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.205456 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.205565 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.227012  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:19.227151  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1821) took 1224 ms. Trace:
I20260709 13:00:19.206837  9442 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.207015  9445 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:19.225594  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:19.207144 10748 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.234658  8892 rpcz_store.cc:276] 0709 13:00:17.997791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.210137 (+1212346us) service_pool.cc:224] Handling call
0709 13:00:19.210540 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.221857 (+ 11317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.222081 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.197093 10612 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:19.206808 10571 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:19.242216  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 183617us
I20260709 13:00:19.227434 10632 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.226516 10662 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:19.263474  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.250094  8891 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:19.235888 10425 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:19.265501  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 184392us
I20260709 13:00:19.237783 10521 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.269107  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.270996  8898 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:19.267289  8748 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:19.275821  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:19.276208  8749 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:19.241894 10748 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:19.268333  8892 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:19.256383  8890 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.258690 10753 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:19.277860  8712 leader_election.cc:336] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.541s (SENT)
I20260709 13:00:19.280817  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1830) took 1232 ms. Trace:
I20260709 13:00:19.253966 10655 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.256233  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:19.277621  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 290652us
I20260709 13:00:19.276906  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.284066  8892 rpcz_store.cc:276] 0709 13:00:18.048171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.242272 (+1194101us) service_pool.cc:224] Handling call
0709 13:00:19.242529 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.280565 (+ 38036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.280761 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.284415  8898 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:19.286173  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 162374us
I20260709 13:00:19.286213  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.290051  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:19.256953 10571 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.283337 10748 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.237692 10661 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.299187  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.300765  8818 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:19.268865 10735 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:00:19.290259 10822 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:00:19.312085 10601 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:19.312825  8712 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.548s (SENT)
I20260709 13:00:19.313509 10655 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.318147  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:19.318886  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.322301  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.329384  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.332916  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:19.330395  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.317130 10753 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.337517  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:19.346938  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 259369us
I20260709 13:00:19.347728  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.303735  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:19.358115  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:19.334846 10601 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:19.334899 10831 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:19.347440 10753 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.357995 10814 tablet_copy_client.cc:323] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.131:44123
I20260709 13:00:19.366354 10833 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:19.366875 10821 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:00:19.368908  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:19.370828  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58075.6us
I20260709 13:00:19.377792 10833 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.378257 10833 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.381330 10481 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.200888  9441 rpcz_store.cc:276] 0709 13:00:17.388827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.740599 (+1351772us) service_pool.cc:224] Handling call
0709 13:00:18.740949 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.741187 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:18.843179 (+101992us) log.cc:844] Serialized 14 byte log entry
0709 13:00:19.108551 (+265372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:19.108631 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:19.108733 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.108837 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.110566 (+  1729us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228218,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":82777,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9197,"thread_start_us":181238,"threads_started":2,"wal-append.queue_time_us":263618}
I20260709 13:00:19.382911 10601 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:19.385784 10825 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:00:19.370561 10753 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.385483  8643 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 3, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:19.381469 10756 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.391004 10756 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:19.390707  8643 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:19.391429 10481 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:19.392234  8643 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 3, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.393560 10481 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:19.393945 10481 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:19.394876 10481 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.396139 10481 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.397465  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.397457 10557 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:19.398319 10571 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.398895 10571 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:19.343591 10612 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.400770  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.353785 10658 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:00:19.403815  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.404421  8740 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:19.399266 10571 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:19.362593 10662 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.412334 10424 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:19.412920 10424 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.413309  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.413939  8749 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:19.413323 10424 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.417695  8643 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.422377  8643 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.424232  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:19.425053  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 87647.9us
I20260709 13:00:19.362563 10591 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.440873  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1513) took 2425 ms. Trace:
I20260709 13:00:19.442869  8742 rpcz_store.cc:276] 0709 13:00:17.015565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.242898 (+1227333us) service_pool.cc:224] Handling call
0709 13:00:18.266191 (+ 23293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:19.001506 (+735315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.001535 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:19.002084 (+   549us) log.cc:844] Serialized 52 byte log entry
0709 13:00:19.104060 (+101976us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:19.104099 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.104185 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:19.440619 (+336434us) raft_consensus.cc:1685] finished
0709 13:00:19.440640 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.440791 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":574676,"raft.queue_time_us":580685,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6521,"thread_start_us":101545,"threads_started":1,"wal-append.queue_time_us":230359}
I20260709 13:00:19.442425  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1202) took 2093 ms. Trace:
I20260709 13:00:19.447409  8747 rpcz_store.cc:276] 0709 13:00:17.356464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.438720 (+1082256us) service_pool.cc:224] Handling call
0709 13:00:18.546071 (+107351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:19.036018 (+489947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.036059 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:19.093233 (+ 57174us) log.cc:844] Serialized 52 byte log entry
0709 13:00:19.323096 (+229863us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:19.425711 (+102615us) log.cc:844] Serialized 14 byte log entry
0709 13:00:19.442008 (+ 16297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.442120 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:19.442177 (+    57us) raft_consensus.cc:1685] finished
0709 13:00:19.442195 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.442355 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":442580,"prepare.queue_time_us":33835,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2521,"raft.queue_time_us":460568,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15806,"thread_start_us":328513,"threads_started":3,"wal-append.queue_time_us":314994}
I20260709 13:00:19.450711  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:19.457687  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 110091us
I20260709 13:00:19.458745  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.418025 10605 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:19.461588  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.404973 10753 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.464284  8896 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:19.473177  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.451148 10656 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.485198  9412 leader_election.cc:400] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:19.461181 10612 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.486176  9412 leader_election.cc:403] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:19.459687 10735 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:19.491158  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 117577us
I20260709 13:00:19.488474  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.513828  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.463454 10831 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:19.520761  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.518724  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.466204 10836 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:19.525323  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62721.1us
I20260709 13:00:19.518460  8747 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:19.526016  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.513682 10612 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.486116 10655 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.527379  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.529955  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:19.519534 10842 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:00:19.533913  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48766.5us
W20260709 13:00:19.552597 10843 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:00:19.524847 10572 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.555011  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.529735 10655 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:19.527160 10836 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.514578 10605 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:00:19.529244 10661 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:19.585781  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59872.6us
I20260709 13:00:19.529934 10612 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.561585 10501 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:00:19.565977 10658 raft_consensus.cc:2764] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:19.566414  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.568132 10587 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.569428  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.569728  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.570281  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.515928 10735 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:19.570700  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.555985  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:19.588219 10655 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.571542 10557 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.697959  8740 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:19.613078  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:19.592767  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 106631us
I20260709 13:00:19.541697 10833 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:19.680833  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 142079us
I20260709 13:00:19.697849  8747 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 1.
I20260709 13:00:19.583307 10832 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:19.556361 10839 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:19.698834  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 232671us
I20260709 13:00:19.683460  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.574330 10591 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.667084 10649 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.709611  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:19.694922 10774 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.585556 10836 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.700908  8744 raft_consensus.cc:2376] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6. Re-sending same reply.
W20260709 13:00:19.711874  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125389us
I20260709 13:00:19.668478 10846 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:19.748265 10849 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:19.753643  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.753990 10752 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:19.754779  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:19.755885 10437 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.001s
I20260709 13:00:19.756353  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.756304 10437 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.757498  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.743752  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.744427  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:19.753268  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.758524  8824 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.615391 10572 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:19.745328  8817 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:19.755810  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.713852  8741 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.747797  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.740868  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:19.760267  8749 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:19.714738 10425 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.714787  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.783581  8744 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:19.786283  8748 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:19.788213 10182 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.741694 10753 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:19.788672 10182 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.711374 10557 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.778548 10572 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:19.792219  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51483.8us
I20260709 13:00:19.793087  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:19.793969  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.684309 10831 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:19.801827  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 92322.8us
I20260709 13:00:19.777426 10849 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:19.757058 10424 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.814137 10617 raft_consensus.cc:2764] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:19.816041 10424 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.784042 10846 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:19.817790 10571 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.818917 10571 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.817926 10846 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.821488  8741 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:19.822104  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.823783  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:19.824844 10656 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.825789 10656 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:19.828390 10661 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.808113 10849 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.810827 10587 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.806175 10591 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.829741 10591 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.806921  8824 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:19.832511  8897 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:19.828212  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:19.836900  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:19.829133  8743 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:19.840579  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1819) took 2428 ms. Trace:
I20260709 13:00:19.758392 10774 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.839754  8742 raft_consensus.cc:2376] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3. Re-sending same reply.
W20260709 13:00:19.802170  8890 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.001s
I20260709 13:00:19.839279  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.840969  8897 rpcz_store.cc:276] 0709 13:00:17.411692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.203904 (+1792212us) service_pool.cc:224] Handling call
0709 13:00:19.832147 (+628243us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:19.832320 (+   173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.840310 (+  7990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.840518 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20710656}
I20260709 13:00:19.850714  8824 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.831972 10591 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.857381  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:19.839555 10424 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.842897 10587 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:19.861035  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 101044us
I20260709 13:00:19.868433  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:19.869714 10189 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:19.869458 10572 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.870143 10189 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.870507 10189 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.870826 10572 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.870985 10774 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.872790 10557 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.867656 10587 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:19.873723 10587 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:19.874203 10632 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 13:00:19.874619 10632 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:19.876847  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 581929us
I20260709 13:00:19.852286 10661 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.881990  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:19.883919  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.860554 10424 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:19.890707  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 363536us
I20260709 13:00:19.893615  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.875947  8890 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:19.899649  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.897073 10424 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.882514 10481 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.900233 10856 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:19.900383 10748 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.001s
I20260709 13:00:19.883174 10661 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:19.896137 10591 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.899361 10554 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.904196 10481 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:19.947533  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1813) took 2537 ms. Trace:
I20260709 13:00:19.947856 10481 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.948153  8890 rpcz_store.cc:276] 0709 13:00:17.409878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.192366 (+1782488us) service_pool.cc:224] Handling call
0709 13:00:19.241825 (+ 49459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.903259 (+661434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.947437 (+ 44178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:19.950146 10756 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:19.904609 10856 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:19.950994 10752 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:19.952739  8824 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:19.953095  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.952315 10856 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.906037 10847 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:00:19.955791 10847 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:19.955572 10655 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:19.956149 10847 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:19.907523 10589 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.003s
I20260709 13:00:19.958041 10655 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:19.910084 10850 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:00:19.958537  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 450) took 3317 ms. Trace:
I20260709 13:00:19.915606  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:19.915957  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.915735  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.916064  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.916355  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:19.916747  8822 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:20.023296  8820 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:19.969648  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84936.6us
W20260709 13:00:19.960600  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 79627.1us
I20260709 13:00:19.917135  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:19.919068  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:19.927124  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:19.951279 10756 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:20.025473  8893 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.942225 10554 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.027844 10554 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:19.919752  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.029613  9449 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.031172  8824 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:19.940177  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.954265  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:20.032965  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.033790  9447 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:19.926157  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.023423  8821 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.035202  8895 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:19.943804  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:20.036551  8897 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.918027  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.043881  9448 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.044788  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:19.917188 10844 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:00:20.192260  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 147623us
I20260709 13:00:20.191768  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:19.929844 10855 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.928656  8908 tablet_copy_service.cc:140] P 7fd110c312c14e17a2bbe6d453493cf2: Received BeginTabletCopySession request for tablet fe13736604454be3b391b8e946192961 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:40708)
I20260709 13:00:19.920416  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:19.917877 10437 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:19.941643 10591 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:20.206832 10425 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:19.965684 10605 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.244s
I20260709 13:00:20.210471 10538 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.210999 10538 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:19.917105 10424 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.211941 10538 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.928874 10831 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.213440 10424 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:20.213506 10831 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.946718 10836 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.916605 10735 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:20.216988 10836 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.918159  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:19.944213  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:20.200917  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.228411  9448 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.201303  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:19.918780  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:20.235443  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 211210us
I20260709 13:00:20.204718  8908 tablet_copy_service.cc:161] P 7fd110c312c14e17a2bbe6d453493cf2: Beginning new tablet copy session on tablet fe13736604454be3b391b8e946192961 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:40708: session id = 112105225cc140459688be5d31dcb071-fe13736604454be3b391b8e946192961
I20260709 13:00:20.206130  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.217118  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.217238  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.245385  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.217728 10831 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:19.921691 10833 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:19.918417 10661 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.903156 10860 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.025574 10756 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:20.225397  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.941458 10837 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.288s
I20260709 13:00:19.942996  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.227394  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:19.921809 10557 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.277s
I20260709 13:00:20.231913  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:19.958099 10589 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.273586  8746 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:20.277691 10189 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.278858  8819 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:20.279899 10189 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:20.024356  9442 rpcz_store.cc:276] 0709 13:00:16.641234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.661563 (+2020329us) service_pool.cc:224] Handling call
0709 13:00:18.723585 (+ 62022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:19.242196 (+518611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.958262 (+716066us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400661a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:19.958451 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072070016}
I20260709 13:00:20.282226  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.283084  9442 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.283951  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.194288 10612 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.256357 10655 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.285418 10655 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.255039 10649 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.286645 10649 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.287016 10649 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.287768 10655 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:20.288432  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.252581  9444 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.233057  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:20.256091 10860 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:20.291074 10860 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.268085  8823 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:20.292266 10756 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.293241  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:20.293793  8823 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.293622  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:20.294678  9447 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.295528 10424 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.295781 10589 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:20.295650  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.296514  9447 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:20.302279  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:20.303222  8815 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:20.303663 10587 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.304158  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:20.305819 10587 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:20.311703 10557 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:20.312567 10557 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:20.313473 10589 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:20.314122  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.314999  8743 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:20.315254 10589 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.315927 10589 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:20.319181  8747 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:20.319250  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.319736 10856 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.322033  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.322815  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.247112  8816 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:20.323149  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.323879  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.324345  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.324683  8744 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:20.324940  8745 raft_consensus.cc:2376] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6. Re-sending same reply.
I20260709 13:00:20.325489  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.322070  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.326304  8744 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:19.956990 10847 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:19.906401 10748 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.326521  8748 raft_consensus.cc:2376] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5. Re-sending same reply.
W20260709 13:00:20.326925 10501 consensus_queue.cc:1175] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 6 } }
I20260709 13:00:19.943392  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.327443  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.328171 10748 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:20.219872 10591 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:20.328459  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.328747  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.328459  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.242727 10774 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:20.336041  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.203267  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:20.248325 10425 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:20.326123  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.191671 10753 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:20.250813  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.270174  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.244431  8892 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:20.329775  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.042502  8893 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:20.199690 10855 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.375039  8818 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.923319 10632 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:20.335209  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.325593  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.336921  8748 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:20.336862  8820 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:20.329339  9445 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.386510  8820 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:20.344341  8897 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:20.355532  9449 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:20.242945 10656 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:20.368157  9448 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:20.379753  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.388427  8744 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.380319  8823 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:20.382735  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.389468  8742 raft_consensus.cc:2376] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6. Re-sending same reply.
I20260709 13:00:20.335959 10774 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:20.384745  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62820.2us
I20260709 13:00:20.374929 10756 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:20.404549  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.384764  8818 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:20.405925  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.407667  8820 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:20.408284  9448 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.409420  9448 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 3 LEARNER]: Advancing to term 4
I20260709 13:00:20.316995 10589 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.410033  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.411528  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.386327  9441 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.414001  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.417484  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.418442  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.418728  8742 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:20.367745 10424 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.405469 10591 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:20.424758  9449 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:20.379302 10753 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.410650 10554 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:20.426084  8744 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 1.
I20260709 13:00:20.417531 10752 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:20.505363  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 461228us
I20260709 13:00:20.417996 10571 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:20.502777  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.500095  8743 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:00:20.510783  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.516s (SENT)
I20260709 13:00:20.499011 10591 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.498622 10856 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:20.585206  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82536us
I20260709 13:00:20.575372  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:20.505841  9442 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:00:20.572610  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70652.1us
I20260709 13:00:20.505503 10752 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.621781 10752 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.629267 10847 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.630443 10662 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.410137 10656 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:20.619273  8695 auto_rebalancer.cc:629] Failed to schedule move for tablet 011f1e8a4b474eddb6b217393cbdd488: Illegal state: Replica 7fd110c312c14e17a2bbe6d453493cf2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:00:20.621248  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 258) took 1569 ms (client timeout 2015 ms). Trace:
W20260709 13:00:20.617986 10864 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:00:20.667961  8746 rpcz_store.cc:269] 0709 13:00:19.051838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.131336 (+ 79498us) service_pool.cc:224] Handling call
0709 13:00:20.621052 (+1489716us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000b6654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:20.621185 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625323264}
I20260709 13:00:20.679956  9442 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:20.696198  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.702481  8744 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:20.507325 10571 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.684758  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.707998  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.631619  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.598882 10589 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.602520 10605 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:20.724853  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:20.725382  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.726877  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:20.800727  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:20.729005 10841 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.169s	user 0.000s	sys 0.011s
I20260709 13:00:20.725101 10837 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:20.768652 10571 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:20.611883 10591 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.819024  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.535087 10554 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:20.824767  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98021.2us
I20260709 13:00:20.666949 10860 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:20.826584  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 101874us
I20260709 13:00:20.725139  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.829486  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.725378  9445 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:20.731453  8816 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:20.849632  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:20.826237  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 315374us
I20260709 13:00:20.798660 10189 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.831822 10837 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.860010  9412 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.860078  8746 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:20.769564  8745 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:20.866000  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:20.863854  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:20.839861 10831 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.827692 10605 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:20.881420  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 333) took 2303 ms (client timeout 2117 ms). Trace:
I20260709 13:00:20.883201  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.885058  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:20.859243  9441 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:20.886693  9448 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 4 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 13:00:20.891225  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.891993  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:20.893200  8896 rpcz_store.cc:269] 0709 13:00:18.579372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.034415 (+455043us) service_pool.cc:224] Handling call
0709 13:00:20.881132 (+1846717us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54001806d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:20.881327 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966711680}
W20260709 13:00:20.898265  9414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.041s	sys 0.049s
I20260709 13:00:20.894845  9448 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 2
I20260709 13:00:20.901078  9448 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:20.882050 10189 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:20.885393 10837 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.893482  8746 raft_consensus.cc:2376] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3. Re-sending same reply.
I20260709 13:00:20.903971  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:20.909003  9448 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:20.887629 10521 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.922888  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:20.891721 10481 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.907290  9414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.041s	sys 0.049s
I20260709 13:00:20.916355  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.915889  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.926159  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.915961  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.926892  8892 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:20.884461 10774 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:20.914222  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.914723  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.891597 10655 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.927901  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 595652us
I20260709 13:00:20.932454  8895 raft_consensus.cc:2376] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3. Re-sending same reply.
I20260709 13:00:20.928239  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.897109 10572 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.930858  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:20.926071  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:20.932610  8893 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:20.898983 10831 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:20.877617 10852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.003s	sys 0.014s
W20260709 13:00:20.909173 10425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.017s	sys 0.063s
W20260709 13:00:20.924754 10839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.019s	sys 0.029s
I20260709 13:00:20.936038  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:20.919458 10189 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.907487 10662 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.923734 10587 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.940977  8898 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:20.926092 10748 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:20.933966  9440 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:20.907768 10867 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:20.944185 10840 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.950255  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:20.934237 10481 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.935547 10605 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:20.936658 10521 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:20.940045 10424 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:20.940511  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.007920  8898 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:20.993613  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70073.2us
I20260709 13:00:21.010044  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:20.976338 10649 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.689s	user 0.019s	sys 0.002s
W20260709 13:00:20.976374 10839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.020s	sys 0.035s
I20260709 13:00:20.942159 10855 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:20.976060 10425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.017s	sys 0.071s
W20260709 13:00:20.954836 10852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.006s	sys 0.014s
I20260709 13:00:20.937805 10655 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:20.979732  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:21.005628  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.007879 10182 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:20.939616 10832 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:00:20.987213 10735 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:21.015491  8908 tablet_copy_source_session.cc:215] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:20.985365  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:20.992570  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.076116 10182 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.088248 10572 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:21.088848 10882 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:21.090291  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.091506  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.092485  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.092970  8740 raft_consensus.cc:2376] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5. Re-sending same reply.
I20260709 13:00:21.093613  8889 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
W20260709 13:00:21.095445  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 85540.7us
I20260709 13:00:21.096196  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 524) took 1177 ms. Trace:
I20260709 13:00:21.096267  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.097824  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.099047  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.099208  8908 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:40708 (request call id 338) took 1171 ms. Trace:
I20260709 13:00:20.987228 10587 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.076843 10182 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.021245  9441 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.100920  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.018195 10656 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.096740  9440 rpcz_store.cc:276] 0709 13:00:19.918370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.918530 (+   160us) service_pool.cc:224] Handling call
0709 13:00:21.096006 (+1177476us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:21.096131 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1501016320}
I20260709 13:00:21.019299 10839 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:21.029577 10554 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:21.130117 10182 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.025365 10425 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:21.132345  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.099637  8908 rpcz_store.cc:276] 0709 13:00:19.928020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.928400 (+   380us) service_pool.cc:224] Handling call
0709 13:00:21.099113 (+1170713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:21.100894 10753 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.165491  8740 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.024905 10649 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:21.166752  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65913us
I20260709 13:00:21.174157  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:21.166800  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 263019us
I20260709 13:00:21.103264 10840 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.104312  8891 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:21.089254 10572 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.017218 10831 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.175604  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.157083  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.131672 10860 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.129113 10867 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:21.162672 10882 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:21.193753  8741 raft_consensus.cc:2368] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4.
I20260709 13:00:21.172374  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 462) took 1236 ms. Trace:
I20260709 13:00:21.221302 10733 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:21.221405  9441 rpcz_store.cc:276] 0709 13:00:19.934607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.938037 (+1003430us) service_pool.cc:224] Handling call
0709 13:00:20.999098 (+ 61061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.132465 (+133367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.171203 (+ 38738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:21.223513  9441 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:21.225531  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:21.225979  8745 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.226447  9440 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:21.225023 10649 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.175896 10617 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.247433 10617 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.169703 10521 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.238538 10867 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.254371  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.255321  8748 raft_consensus.cc:2368] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
I20260709 13:00:21.256211  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.231212  8889 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.179780 10538 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.243328  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.273591  8895 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:21.241641 10870 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
I20260709 13:00:21.263814  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.272367  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.291169  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:21.292121 10878 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:21.292686 10878 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:21.279904 10849 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.294458 10849 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.296056  8889 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.298849  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:21.311560  9442 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:21.312743 10834 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.754s	user 0.006s	sys 0.013s
I20260709 13:00:21.314826  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1222) took 1070 ms. Trace:
I20260709 13:00:21.315328  8741 rpcz_store.cc:276] 0709 13:00:20.244584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.245038 (+   454us) service_pool.cc:224] Handling call
0709 13:00:21.314540 (+1069502us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b5400359154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:21.314747 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162857344}
I20260709 13:00:21.315883  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.316732  8742 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:21.316943 10882 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:21.318110 10875 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:00:21.319535  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 314) took 1064 ms. Trace:
I20260709 13:00:21.320628 10612 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:21.325831  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.319990  8823 rpcz_store.cc:276] 0709 13:00:20.254673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.403985 (+149312us) service_pool.cc:224] Handling call
0709 13:00:20.404251 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:20.417117 (+ 12866us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:20.419921 (+  2804us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:20.425322 (+  5401us) log.cc:844] Serialized 26 byte log entry
0709 13:00:20.924385 (+499063us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:20.924440 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.924529 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.319285 (+394756us) raft_consensus.cc:1685] finished
0709 13:00:21.319311 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.319466 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":4364,"thread_start_us":402891,"threads_started":1,"wal-append.queue_time_us":518080}
W20260709 13:00:21.345675  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.350019  9449 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.350888  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 456) took 1418 ms. Trace:
I20260709 13:00:21.351262  9449 rpcz_store.cc:276] 0709 13:00:19.932948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.327633 (+1394685us) service_pool.cc:224] Handling call
0709 13:00:21.349770 (+ 22137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.350560 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.350821 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:21.351934  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:21.352780  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.353174 10774 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.353659 10774 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:21.354023 10774 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:21.427501  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59370.6us
I20260709 13:00:21.289547  9441 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:21.426151 10840 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:21.434939 10840 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.437057 10840 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:21.440749 10571 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.582s	user 0.017s	sys 0.004s
I20260709 13:00:21.441303 10571 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:21.441169  8740 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:21.442415  9414 leader_election.cc:400] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5.
W20260709 13:00:21.443156  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 269210us
W20260709 13:00:21.452591  8862 connection.cc:256] RPC call timeout handler was delayed by -1.4482374190000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:00:21.452637  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.443259  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.453097  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:21.446054  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:21.453104  8862 connection.cc:256] RPC call timeout handler was delayed by -1.4059374189999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 13:00:21.446815 10189 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:21.447916 10752 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.826s	user 0.014s	sys 0.021s
W20260709 13:00:21.449163  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 273734us
I20260709 13:00:21.432035 10554 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.465689  8748 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:21.426404 10882 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.466262 10182 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.456211  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:21.472298  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:21.475589  8820 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:21.475649 10189 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.476436  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:21.483716 10182 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.485574  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:21.487151  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:21.488013  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 316) took 4063 ms. Trace:
I20260709 13:00:21.491564  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:21.492451 10189 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.492985 10189 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.493420 10189 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.491338  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:21.488497  8894 rpcz_store.cc:276] 0709 13:00:17.409768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.186834 (+1777066us) service_pool.cc:224] Handling call
0709 13:00:19.208047 (+ 21213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.229986 (+ 21939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.230019 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:19.230612 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:00:19.361140 (+130528us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:19.361181 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.361258 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.312165 (+1950907us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:21.472667 (+160502us) raft_consensus.cc:1685] finished
0709 13:00:21.472695 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.472917 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4083,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"spinlock_wait_cycles":295283968,"thread_start_us":130030,"threads_started":1,"wal-append.queue_time_us":211526}
I20260709 13:00:21.499567  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.450620 10656 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:21.500449  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.501777 10182 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.501884  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:21.465719 10836 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:21.501796  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:21.503505  8820 raft_consensus.cc:2368] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:21.502837  8748 raft_consensus.cc:2376] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4. Re-sending same reply.
I20260709 13:00:21.455793  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.502008  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:21.506361  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.509560  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1244) took 1151 ms. Trace:
I20260709 13:00:21.511866  8748 rpcz_store.cc:276] 0709 13:00:20.358295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.471589 (+1113294us) service_pool.cc:224] Handling call
0709 13:00:21.509486 (+ 37897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:21.471778 10735 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.510692  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.508992  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:21.510456  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 57915.9us
I20260709 13:00:21.513615  8897 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.514186  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.490075 10880 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:21.513469  8820 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:21.491387 10848 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.455458 10752 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:21.519553 10182 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.426795 10425 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.522822  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:21.523453  8743 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:21.523739 10892 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:21.524039 10869 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:21.524359 10881 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:00:21.311115 10839 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.505895 10833 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.508947 10481 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.475742 10554 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.509523 10437 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:21.544086  8824 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:21.533687  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:21.548367 10189 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:21.529276 10591 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.664s	user 0.015s	sys 0.015s
I20260709 13:00:21.552578  8897 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:21.583894  8896 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:21.553884 10189 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.589033  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:21.529608  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:21.510069 10656 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:21.583415 10878 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.586100 10481 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.485980 10882 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.521238 10735 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.589967 10617 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.626448 10893 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.629822  8743 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:21.631184  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.601747 10437 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:21.550935 10833 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.608498  9441 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:21.667161  8743 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:21.667517 10889 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:00:21.670025 10437 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:21.637787 10661 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.308s	user 0.008s	sys 0.028s
W20260709 13:00:21.672804  8890 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.015s
I20260709 13:00:21.672277  8895 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:21.670079  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.594899 10591 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:21.701617  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:21.687600  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.704505  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:21.665040 10589 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.851s	user 0.007s	sys 0.026s
W20260709 13:00:21.687039  9444 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.397s	user 0.015s	sys 0.020s
I20260709 13:00:21.698340  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.625370 10481 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.714033  8749 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:21.705325  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 167890us
I20260709 13:00:21.715590  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.717790  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:21.717947  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 203894us
I20260709 13:00:21.718777 10182 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:21.718999  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.710050  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:21.709146  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 166813us
I20260709 13:00:21.705681  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.719179 10182 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:21.720348 10182 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.720515  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:21.720358  8895 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:21.720954  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.705559  8897 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.721446  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.719151  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.734501  8894 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:21.737018  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.737751  8895 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:21.737030  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.739666  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:21.740520  8742 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:21.742799  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:21.744161  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.744751  8895 raft_consensus.cc:2376] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3. Re-sending same reply.
I20260709 13:00:21.698699 10587 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.701097  8890 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:21.714033 10571 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:21.709880  8889 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:21.720597 10735 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:21.757113  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.760701  8892 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:21.760897  8742 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:21.702972  9442 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:21.726198  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1877) took 1810 ms. Trace:
I20260709 13:00:21.753777  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:21.714071  9444 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:21.704100 10661 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:21.627012 10893 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:21.537607 10848 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:21.772166  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:22.162060 10840 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.014s
W20260709 13:00:22.207762 10840 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.001s	sys 0.014s
W20260709 13:00:22.051857  8745 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.825s	user 0.006s	sys 0.011s
I20260709 13:00:21.786721  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:22.209172  8712 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.508s (SENT)
W20260709 13:00:22.049221 10662 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.067s	user 0.010s	sys 0.033s
W20260709 13:00:22.101715  8788 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.177s (SENT)
I20260709 13:00:22.210733 10591 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:21.769964 10846 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:22.211475  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 492588us
I20260709 13:00:21.749125 10612 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:21.863346 10182 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:22.212736  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 494037us
I20260709 13:00:21.770190 10903 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:22.018978 10837 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.087s	user 0.020s	sys 0.025s
W20260709 13:00:22.052887  8891 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.832s	user 0.005s	sys 0.018s
W20260709 13:00:22.054519 10756 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.640s	user 0.021s	sys 0.001s
I20260709 13:00:21.829510  8742 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.941457  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1252) took 1431 ms. Trace:
I20260709 13:00:22.031080 10656 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:22.208452  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 487264us
I20260709 13:00:22.267509  8746 rpcz_store.cc:276] 0709 13:00:20.824301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.475015 (+650714us) service_pool.cc:224] Handling call
0709 13:00:21.483664 (+  8649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.483908 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:21.765168 (+281260us) log.cc:844] Serialized 14 byte log entry
0709 13:00:21.940941 (+175773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:21.941029 (+    88us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:21.941099 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.941191 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.941375 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6297,"prepare.queue_time_us":396594,"prepare.run_wall_time_us":37,"raft.queue_time_us":112,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4562,"spinlock_wait_cycles":34563840,"thread_start_us":450722,"threads_started":2}
I20260709 13:00:21.756534 10572 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.219376  8745 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:22.285202  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 275) took 1350 ms. Trace:
I20260709 13:00:22.285694  8745 rpcz_store.cc:276] 0709 13:00:20.934640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.099936 (+165296us) service_pool.cc:224] Handling call
0709 13:00:22.285122 (+1185186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.287894  8746 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:22.052744  8741 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:22.231429  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:22.053402  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:22.304592  8740 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:21.787125  8819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1873) took 1873 ms (client timeout 1837 ms). Trace:
I20260709 13:00:22.308447  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1248) took 1799 ms. Trace:
I20260709 13:00:22.308986  8740 rpcz_store.cc:276] 0709 13:00:20.508686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.451837 (+943151us) service_pool.cc:224] Handling call
0709 13:00:22.053234 (+601397us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:22.308354 (+255120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10332032}
I20260709 13:00:21.744513 10860 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.232553  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:22.179653  8893 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:22.233009  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 168681us
I20260709 13:00:22.316636  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1517) took 1377 ms. Trace:
I20260709 13:00:22.317168  8893 rpcz_store.cc:276] 0709 13:00:20.938959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.939110 (+   151us) service_pool.cc:224] Handling call
0709 13:00:22.173350 (+1234240us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:22.173646 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.316322 (+142676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.316568 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4789120}
I20260709 13:00:22.318635  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:22.048964 10538 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.010s
W20260709 13:00:21.910297  8713 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.769s (SENT) [suppressed 72 similar messages]
I20260709 13:00:21.774765 10867 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:21.978791 10752 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:21.880286 10847 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.250s	user 0.015s	sys 0.004s
I20260709 13:00:22.239107 10182 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.785167  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 466) took 1100 ms. Trace:
I20260709 13:00:22.049710  8816 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:22.219686 10554 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:22.215751 10481 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:22.235641 10774 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.881s	user 0.014s	sys 0.021s
I20260709 13:00:22.261837 10893 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.262010 10848 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:21.765180  8815 rpcz_store.cc:276] 0709 13:00:19.915322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.305038 (+389716us) service_pool.cc:224] Handling call
0709 13:00:20.376914 (+ 71876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:20.417809 (+ 40895us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:20.928689 (+510880us) log.cc:844] Serialized 14 byte log entry
0709 13:00:21.168756 (+240067us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.227230 (+ 58474us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.227389 (+   159us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:21.227428 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.227523 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.701582 (+474059us) raft_consensus.cc:1685] finished
0709 13:00:21.703550 (+  1968us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.716806 (+ 13256us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":577043,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":5335,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":10073,"thread_start_us":576149,"threads_started":2,"wal-append.queue_time_us":386362}
I20260709 13:00:21.936664  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:22.057456  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:22.240379  8891 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 2.
I20260709 13:00:21.834511  9445 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.869668 10879 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:21.814538 10735 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:22.059270 10733 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:22.263192  9414 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.912s (SENT)
I20260709 13:00:22.263435  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1254) took 1438 ms. Trace:
I20260709 13:00:21.981521 10900 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:22.232899 10833 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.236406 10662 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.264649  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:22.172811  9440 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:22.267405  8742 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:22.447119  8742 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.448210  8742 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:22.450109  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 185592us
W20260709 13:00:21.851024  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 528) took 1833 ms (client timeout 1933 ms). Trace:
I20260709 13:00:22.238991 10903 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:21.791105  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.319986  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 268) took 1425 ms. Trace:
W20260709 13:00:22.314260  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45304.5us
I20260709 13:00:22.326001  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:22.305665  8819 rpcz_store.cc:269] 0709 13:00:19.914176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.915806 (+  1630us) service_pool.cc:224] Handling call
0709 13:00:21.786872 (+1871066us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540028b754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:21.787035 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586566528}
I20260709 13:00:22.241506 10837 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.341478  8745 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:00:22.451409  9444 rpcz_store.cc:269] 0709 13:00:19.952399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.205831 (+253432us) service_pool.cc:224] Handling call
0709 13:00:21.785567 (+1579736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.453374 10903 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.392716 10189 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.463016 10189 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:22.403636  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 194584us
W20260709 13:00:22.464635  8712 leader_election.cc:336] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.515s (SENT)
I20260709 13:00:22.464798  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:22.465162  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:22.465938  8890 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:22.466147 10893 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:22.414316  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 504265us
I20260709 13:00:22.467108  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:22.340849 10902 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:00:22.340343 10656 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.314824 10572 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.470237  8890 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:22.471051  8898 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:22.471352  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:22.471666 10572 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:22.471807  8712 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.596s (SENT)
I20260709 13:00:22.472242  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:22.472998 10656 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:22.473433  8712 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.603s (SENT)
I20260709 13:00:22.454950  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:22.426995  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.331561 10437 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:00:22.431123  8748 rpcz_store.cc:276] 0709 13:00:20.824880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.512808 (+687928us) service_pool.cc:224] Handling call
0709 13:00:21.530511 (+ 17703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.539134 (+  8623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:21.539181 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.542164 (+  2983us) log.cc:844] Serialized 26 byte log entry
0709 13:00:22.014572 (+472408us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:22.014628 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.014701 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:22.263201 (+248500us) raft_consensus.cc:1685] finished
0709 13:00:22.263228 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.263376 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93213,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":403300,"threads_started":1,"wal-append.queue_time_us":626111}
I20260709 13:00:22.454804  8892 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
I20260709 13:00:22.431334  8895 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:22.422271 10774 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:22.444900  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 181814us
I20260709 13:00:22.451038  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:22.339900  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:22.645795  8893 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:22.340607 10655 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.402024 10658 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.457523  9448 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:22.457649  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 279) took 1326 ms. Trace:
W20260709 13:00:22.664230  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 209459us
I20260709 13:00:22.664309  8745 rpcz_store.cc:276] 0709 13:00:21.131269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.286685 (+1155416us) service_pool.cc:224] Handling call
0709 13:00:22.287063 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.457126 (+170063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.457584 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.453603  8741 rpcz_store.cc:276] 0709 13:00:20.880224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.052013 (+1171789us) service_pool.cc:224] Handling call
0709 13:00:22.052373 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.305095 (+252722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.305377 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.473963  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:22.429378  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1520) took 1439 ms. Trace:
I20260709 13:00:22.392571  9442 rpcz_store.cc:276] 0709 13:00:20.684458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.727471 (+ 43013us) service_pool.cc:224] Handling call
0709 13:00:21.298681 (+571210us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:21.770153 (+471472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144553216}
I20260709 13:00:22.545688 10893 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:22.670776  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 286) took 1059 ms. Trace:
I20260709 13:00:22.671159  8748 rpcz_store.cc:276] 0709 13:00:21.611494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.546831 (+935337us) service_pool.cc:224] Handling call
0709 13:00:22.670327 (+123496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.670484 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.670508 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:22.670543 (+    35us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:22.670574 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.670623 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.670728 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.545557 10656 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.547257  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:22.421824  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 188415us
W20260709 13:00:22.645895  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 195054us
I20260709 13:00:22.656476  8742 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:22.667237  8823 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:22.668323  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:22.678409  8819 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:22.678833  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:22.682103  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
W20260709 13:00:22.682830  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.679909  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:22.668983  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:22.451432 10591 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.673867  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:22.456959 10837 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.680024  8820 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:22.679925  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 263) took 1800 ms. Trace:
I20260709 13:00:22.679287  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:22.678035  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 363839us
I20260709 13:00:22.472869  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:22.663463 10848 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.706103  8893 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:22.687654  8742 rpcz_store.cc:276] 0709 13:00:20.879049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.784892 (+905843us) service_pool.cc:224] Handling call
0709 13:00:21.785317 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.679687 (+894370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.679873 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.710655  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:22.711637  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:22.670969 10572 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.663189 10501 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.663146 10735 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:22.728816  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63547.9us
I20260709 13:00:22.724097  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:22.679085 10655 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.736024  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:22.733595  8862 leader_election.cc:336] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.817s (SENT)
I20260709 13:00:22.737993  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:22.739818  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:22.686003 10658 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.742717 10182 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:22.746658 10916 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.670445  8891 rpcz_store.cc:276] 0709 13:00:20.989973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.992290 (+  2317us) service_pool.cc:224] Handling call
0709 13:00:22.428264 (+1435974us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.747211 10756 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:22.714288  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:22.727548 10840 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.721740 10911 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.726749 10572 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:22.754709 10756 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:22.765865 10774 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.735162 10752 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:22.685745 10656 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.741976 10554 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:22.871949  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 203723us
I20260709 13:00:22.677742 10893 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:22.741540 10867 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:22.873991  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 116459us
I20260709 13:00:22.752383 10921 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.756976  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1854) took 1116 ms. Trace:
I20260709 13:00:22.887388  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:22.885226  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:22.890921  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:22.767905 10733 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.734895 10501 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.747788 10833 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.136s
I20260709 13:00:22.722275 10848 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.877832 10572 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.877116 10756 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.878599 10752 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.770061 10847 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:00:22.890971  8889 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:22.888621 10909 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:22.757695 10837 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.141s
I20260709 13:00:22.908546 10920 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.906035 10554 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.911350 10437 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:22.889076 10916 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:00:22.963948 10916 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:22.964392 10916 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:22.965309 10916 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:22.966677 10916 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.930125 10752 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.981639 10656 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:22.938719 10909 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:22.950064 10554 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.983991 10554 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.984126 10909 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.949618  8815 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:22.890244  8890 rpcz_store.cc:276] 0709 13:00:21.686079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.472577 (+786498us) service_pool.cc:224] Handling call
0709 13:00:22.753973 (+281396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.990644  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:22.991921  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:22.949105 10847 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:22.994524 10847 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:22.995570 10847 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.950133 10920 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:22.996903 10847 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.998608 10920 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:23.003083 10848 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:22.891157 10896 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:00:22.907090 10840 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.985491 10909 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:23.252022 10839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.018s	sys 0.018s
I20260709 13:00:23.081431 10840 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.271224 10437 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:23.571390 10658 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:23.671742 10839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.018s	sys 0.018s
I20260709 13:00:23.749411 10840 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.796103 10893 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:23.868125 10425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.003s	sys 0.015s
W20260709 13:00:23.871212 10182 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.009s	sys 0.011s
W20260709 13:00:23.872967 10182 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.009s	sys 0.012s
W20260709 13:00:23.872041 10848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.000s	sys 0.014s
W20260709 13:00:23.879827 10837 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.002s	sys 0.026s
W20260709 13:00:23.897595 10848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.000s	sys 0.014s
W20260709 13:00:23.897710 10837 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.002s	sys 0.027s
I20260709 13:00:23.870100 10501 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:23.880259 10925 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
W20260709 13:00:23.883021 10903 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.421s	user 0.009s	sys 0.000s
W20260709 13:00:23.899463 10909 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.000s
I20260709 13:00:23.899726 10661 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:23.880610 10437 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:23.872658 10425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.003s	sys 0.016s
W20260709 13:00:23.897164 10572 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.958s	user 0.006s	sys 0.002s
W20260709 13:00:23.883678 10752 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.908s	user 0.007s	sys 0.000s
I20260709 13:00:26.298343 10752 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:23.894146 10756 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.002s
I20260709 13:00:26.302443 10756 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:23.893682 10554 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.909s	user 0.008s	sys 0.000s
I20260709 13:00:26.306418 10554 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:23.899395 10903 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:24.977991  8713 connection.cc:256] RPC call timeout handler was delayed by -2.2562356730000488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 13:00:26.311110  8713 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.945s (SENT)
I20260709 13:00:26.312165 10903 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:26.312734  8713 connection.cc:256] RPC call timeout handler was delayed by -2.0843686680000246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260709 13:00:26.313254  8713 connection.cc:256] RPC call timeout handler was delayed by -1.9795583119999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
W20260709 13:00:26.313644  8713 connection.cc:256] RPC call timeout handler was delayed by -1.7580489360000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
I20260709 13:00:26.313390 10572 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:26.313983  8713 connection.cc:256] RPC call timeout handler was delayed by -1.5893544260000567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 13:00:24.983296  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.315728 10572 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:25.022753 10661 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:25.028293  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 2.55592e+06us
W20260709 13:00:25.033923  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 280) took 3864 ms (client timeout 2112 ms). Trace:
I20260709 13:00:25.041422  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:26.316507  8747 rpcz_store.cc:269] 0709 13:00:21.169363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.033685 (+3864322us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:25.033832 (+   147us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:26.316433 10661 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.317466 10756 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.317546  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 471) took 1276 ms. Trace:
I20260709 13:00:26.301209 10752 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.318181  9443 rpcz_store.cc:276] 0709 13:00:25.040934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.041081 (+   147us) service_pool.cc:224] Handling call
0709 13:00:26.317457 (+1276376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.319000 10661 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.319568 10661 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:26.319814  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.316828 10554 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.320681  9443 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:25.047683 10814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe13736604454be3b391b8e946192961. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:25.051302  8864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.160s	user 0.013s	sys 0.012s
W20260709 13:00:26.325814  8864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.434s	user 0.015s	sys 0.013s
W20260709 13:00:26.326193  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 3.43673e+06us
I20260709 13:00:25.054366 10878 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:26.328624 10554 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:25.072918  8740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.647s	user 0.062s	sys 0.072s
I20260709 13:00:25.076464 10591 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:26.330001 10591 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:26.330873  8740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.905s	user 0.062s	sys 0.072s
I20260709 13:00:26.331948  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 282) took 4894 ms. Trace:
I20260709 13:00:26.332602 10591 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:25.077351  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1241) took 2430 ms. Trace:
W20260709 13:00:26.333640  8864 connection.cc:256] RPC call timeout handler was delayed by -2.6014284630000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:00:25.084818  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.335711  9441 raft_consensus.cc:2376] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5. Re-sending same reply.
I20260709 13:00:26.336596  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 472) took 1295 ms. Trace:
I20260709 13:00:26.336899 10591 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.337049  9441 rpcz_store.cc:276] 0709 13:00:25.041235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.084522 (+ 43287us) service_pool.cc:224] Handling call
0709 13:00:26.336514 (+1251992us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.095778 10849 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:25.097179  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1524) took 3938 ms (client timeout 1877 ms). Trace:
W20260709 13:00:26.338877  8864 connection.cc:256] RPC call timeout handler was delayed by -2.5231284630000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
I20260709 13:00:25.476624 10839 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.500617 10837 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.341593 10837 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:25.619694  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.786199  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1522) took 4759 ms. Trace:
I20260709 13:00:26.344002  8898 rpcz_store.cc:276] 0709 13:00:21.026906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.027053 (+   147us) service_pool.cc:224] Handling call
0709 13:00:21.027388 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.471443 (+1444055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.785864 (+3314421us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54003bd154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:25.786108 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3321336192}
W20260709 13:00:25.786952  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9280 after lost signal to thread 7641
I20260709 13:00:26.345467 10837 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:26.346062  9470 debug-util.cc:398] Leaking SignalData structure 0x7b0800287840 after lost signal to thread 8681
I20260709 13:00:26.346659  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:26.347431  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003ec1e0 after lost signal to thread 8773
I20260709 13:00:26.337585 10591 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:26.348182  8898 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:26.348759  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003c10c0 after lost signal to thread 8848
W20260709 13:00:26.349048  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003c13a0 after lost signal to thread 8849
I20260709 13:00:26.340865 10839 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.350795  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.351107 10839 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:25.858259  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.117s	user 0.091s	sys 0.116s
I20260709 13:00:26.351393  8898 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:26.351812  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.611s	user 0.091s	sys 0.116s
W20260709 13:00:26.352357  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 3.61408e+06us
I20260709 13:00:26.347467  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 471) took 1294 ms. Trace:
I20260709 13:00:26.352835  9443 rpcz_store.cc:276] 0709 13:00:25.052767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.319394 (+1266627us) service_pool.cc:224] Handling call
0709 13:00:26.347397 (+ 28003us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.353839  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.354574  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 475) took 1300 ms. Trace:
I20260709 13:00:26.354928  9443 rpcz_store.cc:276] 0709 13:00:25.053844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.353527 (+1299683us) service_pool.cc:224] Handling call
0709 13:00:26.354494 (+   967us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.859486 10632 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.356904 10591 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:25.860864 10911 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.357600 10911 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.358481  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.359297  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 530) took 1305 ms. Trace:
I20260709 13:00:26.359611 10911 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:25.861687  8818 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.865427  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.360390  8824 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.360481  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1887) took 4865 ms. Trace:
I20260709 13:00:25.866830  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.360919  8818 rpcz_store.cc:276] 0709 13:00:21.494865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.495525 (+   660us) service_pool.cc:224] Handling call
0709 13:00:25.861205 (+4365680us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:25.861480 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.360194 (+498714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.360402 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3540864}
I20260709 13:00:26.361740  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 469) took 1321 ms. Trace:
W20260709 13:00:25.876680  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1504) took 4963 ms (client timeout 1602 ms). Trace:
I20260709 13:00:26.357198 10632 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:26.362426  8896 rpcz_store.cc:269] 0709 13:00:20.913055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.927788 (+ 14733us) service_pool.cc:224] Handling call
0709 13:00:21.325670 (+397882us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:25.876360 (+4550690us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400030cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.876555 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596416640}
W20260709 13:00:25.879648  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.996s	user 0.015s	sys 0.012s
W20260709 13:00:25.904755 10848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.006s	user 0.010s	sys 0.015s
W20260709 13:00:23.893114 10847 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.694s	user 0.006s	sys 0.000s
W20260709 13:00:26.365928  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.482s	user 0.016s	sys 0.013s
W20260709 13:00:25.910003  9446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 522) took 5993 ms (client timeout 2734 ms). Trace:
W20260709 13:00:26.366557  9446 rpcz_store.cc:269] 0709 13:00:19.916685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.916879 (+   194us) service_pool.cc:224] Handling call
0709 13:00:25.909704 (+5992825us) spinlock_profiling.cc:243] Waited 5.76 s on lock 0x7b54001982d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.909867 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13258529024}
I20260709 13:00:26.367318  8818 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:26.367496  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.367880  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.368180  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:26.368574  9446 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:26.368772  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.368957  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 473) took 1326 ms. Trace:
I20260709 13:00:25.914929  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 463) took 5980 ms. Trace:
W20260709 13:00:25.917466  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1840) took 4284 ms (client timeout 1837 ms). Trace:
I20260709 13:00:25.918002 10538 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:26.153193  8895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1849) took 4517 ms (client timeout 1709 ms). Trace:
I20260709 13:00:26.163152  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:26.163862  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 458) took 6233 ms (client timeout 2769 ms). Trace:
W20260709 13:00:26.167943  8788 connection.cc:256] RPC call timeout handler was delayed by -3.2510285050000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:00:26.168884  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:26.169520 10936 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:26.169911 10916 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 3.203s	user 0.036s	sys 0.027s
W20260709 13:00:26.179242 10914 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 3.960s	user 0.022s	sys 0.030s
W20260709 13:00:26.182850 10912 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 3.758s	user 0.009s	sys 0.043s
W20260709 13:00:26.187096  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1223) took 5917 ms (client timeout 2098 ms). Trace:
I20260709 13:00:26.389581 10936 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:26.389945  8749 rpcz_store.cc:269] 0709 13:00:20.269687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.269866 (+   179us) service_pool.cc:224] Handling call
0709 13:00:26.186727 (+5916861us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000b4d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.186966 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3075394944}
I20260709 13:00:26.192432 10833 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.391377 10833 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:26.362102  9442 rpcz_store.cc:276] 0709 13:00:25.040570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.866496 (+825926us) service_pool.cc:224] Handling call
0709 13:00:26.361667 (+495171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.392783 10936 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:26.209646  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.394446 10632 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:26.211897 10840 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 2.345s	user 0.015s	sys 0.014s
I20260709 13:00:26.386003 10756 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:26.395476 10840 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.395766 10756 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.397547 10840 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.215149  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 460) took 6281 ms. Trace:
I20260709 13:00:26.398026 10756 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.398105  9448 rpcz_store.cc:276] 0709 13:00:19.934059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.932684 (+998625us) service_pool.cc:224] Handling call
0709 13:00:20.933053 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.663964 (+1730911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.214862 (+3550898us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:26.215084 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3507135360}
I20260709 13:00:26.399443  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:26.399659  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:26.399978  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1862) took 4687 ms (client timeout 1542 ms). Trace:
W20260709 13:00:26.219316  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 468) took 4744 ms (client timeout 2952 ms). Trace:
W20260709 13:00:26.400446  9449 rpcz_store.cc:269] 0709 13:00:21.475054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.219097 (+4744043us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:26.219227 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:26.401551  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.402727  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.403295  9449 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.220974  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 466) took 4772 ms. Trace:
W20260709 13:00:26.224406  9414 connection.cc:256] RPC call timeout handler was delayed by -1.7384377370000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
I20260709 13:00:26.405580  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.406414  8818 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:26.227936 10939 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.226672 10937 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:26.234591 10658 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:26.236199  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.331s	user 0.017s	sys 0.047s
I20260709 13:00:26.239524 10900 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:26.241313 10890 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 4.542s	user 0.049s	sys 0.059s
I20260709 13:00:26.243180 10885 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:26.417644 10900 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.417901 10885 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.420042 10885 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.420161 10900 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:26.244779  8862 connection.cc:256] RPC call timeout handler was delayed by -2.981208209999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 13:00:26.420637  8862 connection.cc:256] RPC call timeout handler was delayed by -1.1948111290000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33030000000000004s for a response
I20260709 13:00:26.421697  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:26.416121  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.245467 10940 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.246711  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:26.251483  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1535) took 3525 ms (client timeout 2189 ms). Trace:
I20260709 13:00:26.264420 10189 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.280431  8892 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:26.283497  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 278) took 5252 ms. Trace:
I20260709 13:00:26.426394  8746 rpcz_store.cc:276] 0709 13:00:21.030951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.269346 (+1238395us) service_pool.cc:224] Handling call
0709 13:00:22.287533 (+ 18187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.288368 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.283134 (+3994766us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400160dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.283385 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41130752}
W20260709 13:00:26.291769 10521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.683s	user 0.030s	sys 0.087s
I20260709 13:00:26.291675 10943 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:00:26.328684 10878 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:26.420857 10939 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:26.417647 10890 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 4.721s	user 0.050s	sys 0.059s
W20260709 13:00:25.062721  9413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.173s	user 0.002s	sys 0.018s
W20260709 13:00:26.433598  9413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.547s	user 0.002s	sys 0.019s
I20260709 13:00:26.434831  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1863) took 4721 ms. Trace:
I20260709 13:00:26.435231  8892 rpcz_store.cc:276] 0709 13:00:21.713076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.721393 (+1008317us) service_pool.cc:224] Handling call
0709 13:00:22.721785 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.426022 (+3704237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.434578 (+  8556us) spinlock_profiling.cc:243] Waited 3.56 s on lock 0x7b540045a6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:26.434770 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8187199872}
I20260709 13:00:26.436869  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1868) took 4683 ms. Trace:
I20260709 13:00:26.379557 10538 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:26.438123 10538 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.439628 10538 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.333442  8815 rpcz_store.cc:276] 0709 13:00:22.646714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.646816 (+   102us) service_pool.cc:224] Handling call
0709 13:00:25.077104 (+2430288us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540040a454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.077242 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412198784}
I20260709 13:00:26.442226  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.443217  8815 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:26.444118  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.431135 10911 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.445114 10946 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:26.431208 10521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.831s	user 0.030s	sys 0.088s
I20260709 13:00:26.446568 10946 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.334430 10554 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.431311 10752 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.448015 10752 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:25.083632  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.449558 10554 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:26.449919  8816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1240) took 3803 ms (client timeout 1709 ms). Trace:
W20260709 13:00:26.339205  8894 rpcz_store.cc:269] 0709 13:00:21.158392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.499348 (+340956us) service_pool.cc:224] Handling call
0709 13:00:25.096946 (+3597598us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540019e6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.097098 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538189056}
I20260709 13:00:25.098409  8893 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.340091  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.332366  8740 rpcz_store.cc:276] 0709 13:00:21.437598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.310135 (+872537us) service_pool.cc:224] Handling call
0709 13:00:22.332571 (+ 22436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.333031 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.333070 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:22.333621 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.331464 (+3997843us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:26.331524 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.331621 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.331681 (+    60us) raft_consensus.cc:1685] finished
0709 13:00:26.331698 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.331873 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":56256,"thread_start_us":3905903,"threads_started":1,"wal-append.queue_time_us":1025898,"wal-append.run_cpu_time_us":1926,"wal-append.run_wall_time_us":1001963}
I20260709 13:00:26.370007  8822 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:26.370903  8819 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:26.379830  8889 rpcz_store.cc:269] 0709 13:00:21.633496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.057141 (+423645us) service_pool.cc:224] Handling call
0709 13:00:25.917151 (+3860010us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400396a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.917368 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790516096}
I20260709 13:00:26.379920  9446 rpcz_store.cc:276] 0709 13:00:25.042537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.367641 (+1325104us) service_pool.cc:224] Handling call
0709 13:00:26.368907 (+  1266us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.380132  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.371322  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.369112  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:26.379593  8895 rpcz_store.cc:269] 0709 13:00:21.636030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.771844 (+135814us) service_pool.cc:224] Handling call
0709 13:00:26.152903 (+4381059us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b540046da54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.153086 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142439680}
W20260709 13:00:26.380735  9447 rpcz_store.cc:269] 0709 13:00:19.930806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.297221 (+366415us) service_pool.cc:224] Handling call
0709 13:00:26.163637 (+5866416us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b5400740254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.163785 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4743151360}
W20260709 13:00:26.366444  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd9e0 after lost signal to thread 8922
W20260709 13:00:26.384415  8864 connection.cc:256] RPC call timeout handler was delayed by -2.214428463000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:00:26.450409  8896 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:26.450699 10752 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:26.453305  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.453707  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.432334 10878 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.454070  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1865) took 4731 ms. Trace:
I20260709 13:00:26.454594 10878 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.405441  9440 rpcz_store.cc:276] 0709 13:00:21.448990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.450871 (+  1881us) service_pool.cc:224] Handling call
0709 13:00:21.511691 (+ 60820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.446220 (+934529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.220702 (+3774482us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54006a95d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:26.220910 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352643840}
I20260709 13:00:26.455382  8894 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:26.455994  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1869) took 4699 ms. Trace:
W20260709 13:00:26.456068  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003cdce0 after lost signal to thread 9473
W20260709 13:00:26.456493  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003cdb60 after lost signal to thread 9474
I20260709 13:00:26.456331  8894 rpcz_store.cc:276] 0709 13:00:21.756531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.454845 (+4698314us) service_pool.cc:224] Handling call
0709 13:00:26.455099 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.455782 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.455946 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.456889  8895 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:26.457516  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1871) took 4673 ms. Trace:
I20260709 13:00:26.432512 10940 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.457872  8895 rpcz_store.cc:276] 0709 13:00:21.784395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.456352 (+4671957us) service_pool.cc:224] Handling call
0709 13:00:26.456617 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.457280 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.457466 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.458205  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1855) took 4751 ms. Trace:
I20260709 13:00:26.454620  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.458675  8893 rpcz_store.cc:276] 0709 13:00:21.706768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.713704 (+1006936us) service_pool.cc:224] Handling call
0709 13:00:25.098211 (+2384507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.457397 (+1359186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.458015 (+   618us) spinlock_profiling.cc:243] Waited 2.39 s on lock 0x7b540019e6d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.458151 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5485755776}
I20260709 13:00:26.459676  9446 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:26.454488  8896 rpcz_store.cc:276] 0709 13:00:21.722610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.363402 (+4640792us) service_pool.cc:224] Handling call
0709 13:00:26.363691 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.453268 (+ 89577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.453852 (+   584us) spinlock_profiling.cc:243] Waited 86.5 ms on lock 0x7b540015d954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:26.454011 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198966656}
I20260709 13:00:26.392491 10833 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.393047  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.461159 10833 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.461802  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.462986  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.463436  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:26.463582  9442 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:26.464236  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 531) took 1409 ms. Trace:
I20260709 13:00:26.464488  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.464614  9442 rpcz_store.cc:276] 0709 13:00:25.054381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.462725 (+1408344us) service_pool.cc:224] Handling call
0709 13:00:26.464172 (+  1447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.421367 10937 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.468752  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.470073  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.470609 10937 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:26.217237 10846 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.471391 10846 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.473474  9448 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:26.474920  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 472) took 1421 ms. Trace:
I20260709 13:00:26.475239  9448 rpcz_store.cc:276] 0709 13:00:25.053061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.399389 (+1346328us) service_pool.cc:224] Handling call
0709 13:00:26.474757 (+ 75368us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.474861 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168388480}
I20260709 13:00:26.476372  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:26.408130  8897 rpcz_store.cc:269] 0709 13:00:21.712792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.736686 (+ 23894us) service_pool.cc:224] Handling call
0709 13:00:26.168737 (+4432051us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:26.399911 (+231174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5745536}
I20260709 13:00:26.341594 10947 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:26.422880  8818 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3. Re-sending same reply.
W20260709 13:00:26.385484 10848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.487s	user 0.010s	sys 0.015s
I20260709 13:00:26.379665  9445 rpcz_store.cc:276] 0709 13:00:19.934881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.293194 (+1358313us) service_pool.cc:224] Handling call
0709 13:00:21.770064 (+476870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.428317 (+658253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.914640 (+3486323us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54006fe0d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:25.914827 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085825280}
I20260709 13:00:26.431249 10658 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.370409  8821 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:26.459584 10940 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.465234  8822 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
W20260709 13:00:26.425609  8891 rpcz_store.cc:269] 0709 13:00:22.726003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.251264 (+3525261us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:26.251393 (+   129us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:26.431618  8817 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:26.389956  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 470) took 1349 ms. Trace:
I20260709 13:00:26.501590  8643 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 196) took 9958 ms. Trace:
I20260709 13:00:26.201064 10882 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.456099  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.460704 10189 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:26.523983 10189 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.454924  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.525444  9447 raft_consensus.cc:2376] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4. Re-sending same reply.
I20260709 13:00:26.460587  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.526863  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.527853  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.466271  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 473) took 1413 ms. Trace:
I20260709 13:00:26.466365  8820 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.465670  8823 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:26.437194  8892 rpcz_store.cc:276] 0709 13:00:21.753158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.436097 (+4682939us) service_pool.cc:224] Handling call
0709 13:00:26.436396 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.436583 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.436606 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.436644 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.436682 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.436731 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.436820 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.465375  8864 connection.cc:256] RPC call timeout handler was delayed by -1.0885284630000456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33030000000000004s for a response
I20260709 13:00:26.409749  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:00:26.536550  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1244) took 3888 ms (client timeout 2052 ms). Trace:
I20260709 13:00:26.536984  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:26.536929  8824 rpcz_store.cc:269] 0709 13:00:22.665972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.666527 (+   555us) service_pool.cc:224] Handling call
0709 13:00:25.865221 (+3198694us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:26.536482 (+671261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45002112}
I20260709 13:00:26.537994  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.538540  8824 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:26.430783  9449 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
W20260709 13:00:26.387964  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71337us
W20260709 13:00:26.539433  8712 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.662s (SENT)
W20260709 13:00:26.453997  8788 connection.cc:256] RPC call timeout handler was delayed by -2.870874472999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 13:00:26.539880  8788 connection.cc:256] RPC call timeout handler was delayed by -2.7718744729999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:00:26.540239  8788 connection.cc:256] RPC call timeout handler was delayed by -2.6954069669999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 13:00:26.540606  8788 connection.cc:256] RPC call timeout handler was delayed by -2.52202850499998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25510000000000005s for a response
W20260709 13:00:26.541122  8788 connection.cc:256] RPC call timeout handler was delayed by -2.161128505000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2952s for a response
I20260709 13:00:26.540005  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:26.541694  8788 connection.cc:256] RPC call timeout handler was delayed by -1.9148628779999513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
W20260709 13:00:26.487962  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 193613us
W20260709 13:00:26.489326  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 96399.2us
I20260709 13:00:26.480939  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:26.546800  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65958.5us
W20260709 13:00:26.493360  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.588s	user 0.018s	sys 0.050s
W20260709 13:00:26.549099  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 3.64557e+06us
I20260709 13:00:26.494057  8894 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:26.495767  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.551008  9448 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:26.496522  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:26.502087  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:26.552539  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 56133.1us
I20260709 13:00:26.553478  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:26.501586  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 94954.4us
I20260709 13:00:26.511286 10945 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:26.520607 10943 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.359612  9443 rpcz_store.cc:276] 0709 13:00:25.054115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.358206 (+1304091us) service_pool.cc:224] Handling call
0709 13:00:26.359238 (+  1032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.533435 10189 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:26.450385  8816 rpcz_store.cc:269] 0709 13:00:22.646374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.654027 (+  7653us) service_pool.cc:224] Handling call
0709 13:00:25.083502 (+2429475us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:26.449840 (+1366338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42913024}
W20260709 13:00:26.554553  8863 connection.cc:256] RPC call timeout handler was delayed by -4.1074899549999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.325s for a response
I20260709 13:00:26.549875  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:26.470634  9442 raft_consensus.cc:2376] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6. Re-sending same reply.
W20260709 13:00:26.576216  8789 connection.cc:256] RPC call timeout handler was delayed by -2.3737036599999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 13:00:26.576884  8863 connection.cc:256] RPC call timeout handler was delayed by -3.304998841999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28250000000000003s for a response
W20260709 13:00:26.578898  8789 connection.cc:256] RPC call timeout handler was delayed by -2.3656036599999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:00:26.573742  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.579298  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.579566  9447 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:26.580060  8818 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:00:26.579306  8789 connection.cc:256] RPC call timeout handler was delayed by -2.30980366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 13:00:26.573185  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.580814  9449 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:26.572949  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:26.580544  8789 connection.cc:256] RPC call timeout handler was delayed by -2.2423036600000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 13:00:26.582106  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.582614  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.523600  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.583285  8824 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:26.581115  8818 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:26.584959  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:26.577149  8715 leader_election.cc:400] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:26.585856  8715 leader_election.cc:403] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:26.572016  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1533) took 3846 ms. Trace:
I20260709 13:00:26.586629 10189 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.586493  8894 rpcz_store.cc:276] 0709 13:00:22.725345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.493410 (+3768065us) service_pool.cc:224] Handling call
0709 13:00:26.493747 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.550398 (+ 56651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.571956 (+ 21558us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.585994  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:26.521893  8643 rpcz_store.cc:276] 0709 13:00:16.464004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.469229 (+  5225us) service_pool.cc:224] Handling call
0709 13:00:26.421916 (+9952687us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:19.875189 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:19.914024 (+ 38835us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:19.914076 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:00:19.914096 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:19.915039 (+   943us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:19.915144 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:19.915174 (+    30us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:00:19.915858 (+   684us) tablet.cc:701] Row locks acquired
0709 13:00:19.915912 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:00:19.916078 (+   166us) write_op.cc:270] Start()
0709 13:00:19.916232 (+   154us) write_op.cc:276] Timestamp: P: 1783602019916006 usec, L: 0
0709 13:00:19.916257 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:20.294130 (+377873us) log.cc:844] Serialized 2452 byte log entry
0709 13:00:21.610295 (+1316165us) op_driver.cc:464] REPLICATION: finished
0709 13:00:22.054341 (+444046us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cae28 00007f90f9f05581 00007f90f9f04d58 00007f90f9f099f1 00007f90f9f099a9 00007f90f9f09939 00007f90f9f09901 00007f90f9f08b2d 000000000038f354 000000000038f199
0709 13:00:26.172561 (+4118220us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f9f05ef0
0709 13:00:26.238070 (+ 65509us) write_op.cc:301] APPLY: starting
0709 13:00:26.238169 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:26.238369 (+   200us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:26.238404 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:26.239455 (+  1051us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:26.239837 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:00:26.239888 (+    51us) write_op.cc:312] APPLY: finished
0709 13:00:26.240231 (+   343us) log.cc:844] Serialized 53 byte log entry
0709 13:00:26.255893 (+ 15662us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:26.256018 (+   125us) write_op.cc:454] Released schema lock
0709 13:00:26.256083 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:26.432931 (+176848us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:26.432963 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1520065,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":4615078,"apply.run_cpu_time_us":3551,"apply.run_wall_time_us":18429,"num_ops":6,"prepare.queue_time_us":145815,"prepare.run_cpu_time_us":41172,"prepare.run_wall_time_us":2188792,"raft.queue_time_us":248798,"raft.run_cpu_time_us":110476,"raft.run_wall_time_us":4890679,"replication_time_us":1672310,"spinlock_wait_cycles":97194112,"thread_start_us":5745526,"threads_started":5,"wal-append.queue_time_us":661240,"wal-append.run_cpu_time_us":14014,"wal-append.run_wall_time_us":1600518}]]}
I20260709 13:00:26.581887  8823 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:26.590529  8818 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:26.581943  8789 connection.cc:256] RPC call timeout handler was delayed by -2.23960366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
W20260709 13:00:26.620790  8789 connection.cc:256] RPC call timeout handler was delayed by -2.1820036600000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:00:26.574319  8892 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:00:26.604379  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:26.607575  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.583179  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.624985  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:26.522403 10937 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 13:00:26.534437  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1243) took 3887 ms (client timeout 1622 ms). Trace:
I20260709 13:00:26.627728  9446 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:00:26.605446  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 175884us
W20260709 13:00:26.632249  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.112s (SENT)
I20260709 13:00:26.628285  9447 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:26.632872 10950 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:26.633204  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 474) took 1579 ms. Trace:
I20260709 13:00:26.633586  9447 rpcz_store.cc:276] 0709 13:00:25.053608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.580166 (+1526558us) service_pool.cc:224] Handling call
0709 13:00:26.628116 (+ 47950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.632962 (+  4846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.633148 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.627601  8896 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:26.634315 10950 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:26.634786 10878 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:26.635680 10878 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.636075 10878 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:26.642468  9449 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:26.577476 10943 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:26.572640 10752 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.591022 10521 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:26.642294  8863 leader_election.cc:336] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.825s (SENT)
I20260709 13:00:26.666623  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.575477 10882 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:26.643148  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 103792us
I20260709 13:00:26.619151  9440 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:26.678383  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.679266  8746 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:26.679738  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.680338  8749 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:26.576144 10939 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.571669 10903 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:26.576046 10833 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.521353  9444 rpcz_store.cc:276] 0709 13:00:25.040770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.162818 (+1122048us) service_pool.cc:224] Handling call
0709 13:00:26.389888 (+227070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.690773 10878 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.575804 10879 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.578557 10856 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:26.651612  9415 leader_election.cc:336] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.554s (SENT)
I20260709 13:00:26.722623  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.725713  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.726004  8741 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:26.726544  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 75027.8us
I20260709 13:00:26.582607 10909 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.534184  9441 rpcz_store.cc:276] 0709 13:00:25.053353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.338295 (+1284942us) service_pool.cc:224] Handling call
0709 13:00:26.466212 (+127917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.580549 10846 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.572453 10940 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.619796 10501 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:26.590003  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:26.632503  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46600.5us
W20260709 13:00:26.660447  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 115918us
I20260709 13:00:26.668788  8820 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:26.668910  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91937.4us
I20260709 13:00:26.730479  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.731621 10957 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:26.731964  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:26.732620  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.627868 10936 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:00:26.733065  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:26.637356  8817 rpcz_store.cc:269] 0709 13:00:22.647386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.654551 (+  7165us) service_pool.cc:224] Handling call
0709 13:00:26.246496 (+3591945us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:26.528844 (+282348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10913152}
I20260709 13:00:26.658908 10752 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.736605 10958 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:26.734465 10879 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.738291 10958 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.738662 10958 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.738991 10958 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.739156 10879 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.739933  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.740712  8896 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:26.741425 10538 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:26.741607  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.735488 10937 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:26.682534  8890 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:26.742827  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.743306  8894 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.744406  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.744856  9445 raft_consensus.cc:2376] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4. Re-sending same reply.
I20260709 13:00:26.745565  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.746474  8897 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:26.747162  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.747886  8897 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:26.748844  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:26.749411  8892 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.729056 10856 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.751094 10752 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:26.751683 10957 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:26.752746 10957 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.753348 10957 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.753511 10501 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:26.753756 10957 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:26.753942  8893 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.729133 10632 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.754794 10538 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:26.735829 10940 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:26.755290  8893 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:26.755699 10538 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.758147  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.666833  8891 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:26.580883 10945 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:26.685490  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48465.1us
W20260709 13:00:26.767256  8863 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 3.250s (SENT)
I20260709 13:00:26.767688  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.686335 10955 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:26.689915 10946 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.501885 10947 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.620151 10953 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:00:26.693887  8823 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:26.698019 10658 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:26.699025  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:26.702440  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.780985  8745 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:26.702629  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.703420  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.709273  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.783497  8746 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
W20260709 13:00:26.708302  8712 connection.cc:256] RPC call timeout handler was delayed by -4.4904973390000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
W20260709 13:00:26.784334  8712 connection.cc:256] RPC call timeout handler was delayed by -4.4796973389999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
I20260709 13:00:26.784386  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:26.784922  8712 connection.cc:256] RPC call timeout handler was delayed by -4.4490973389999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
I20260709 13:00:26.785306  8889 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
W20260709 13:00:26.621254  8789 connection.cc:256] RPC call timeout handler was delayed by -2.1586036599999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260709 13:00:26.715862  8749 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:00:26.786481  8789 connection.cc:256] RPC call timeout handler was delayed by -1.8355036599999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
W20260709 13:00:26.786934  8789 connection.cc:256] RPC call timeout handler was delayed by -1.4530036599999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25510000000000005s for a response
I20260709 13:00:26.716813  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:26.757071 10940 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:26.788473  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54609.8us
I20260709 13:00:26.668524 10554 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.721760  8743 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:26.717538 10521 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:00:26.722579 10839 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:00:26.730866  8788 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.868s (SENT)
I20260709 13:00:26.721796 10939 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:00:26.758091  9415 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.613s (SENT)
I20260709 13:00:26.683130 10814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:58306 (request call id 505) took 8402 ms. Trace:
I20260709 13:00:26.760110  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:26.836339  8894 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.764722  8892 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:26.768652  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.714102 10903 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.773332  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:26.785426  8712 connection.cc:256] RPC call timeout handler was delayed by -4.4022973390000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 13:00:26.841217  8712 connection.cc:256] RPC call timeout handler was delayed by -4.39059733900001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 13:00:26.711335 10189 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:26.787320  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0921036600000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2952s for a response
I20260709 13:00:26.668788 10882 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:26.788198  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 203619us
I20260709 13:00:26.759173 10661 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:26.793947  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 95046.5us
I20260709 13:00:26.886519 10968 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:26.888085  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.888824  8893 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:26.889925  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.890666  8893 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:26.891952 10961 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:26.892405 10961 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:26.893359 10961 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.894661 10878 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.806285  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.897571  9449 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:26.807508  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.898072  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.898573  9448 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:26.898782  9444 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:26.823071  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 92153.5us
I20260709 13:00:26.729892 10909 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:26.899577  8788 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.153s (SENT)
I20260709 13:00:26.899973  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.900846 10909 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.832368  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:26.901759  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69498.7us
I20260709 13:00:26.901948 10946 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:26.902931 10953 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:26.903669  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:26.833601  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:26.883533  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1864) took 5170 ms. Trace:
W20260709 13:00:26.855882  8789 leader_election.cc:336] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.677s (SENT)
I20260709 13:00:26.794591 10955 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:26.857236  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.911381  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:26.911875  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 153886us
I20260709 13:00:26.858014 10947 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.835616 10846 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:00:26.860101 10945 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.835906  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.863425 10903 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.863903  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:26.821655 10752 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:26.866629  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1534) took 4141 ms (client timeout 2126 ms). Trace:
I20260709 13:00:26.867359 10501 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.868700 10856 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:26.754623 10957 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:26.805626 10900 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.805115 10879 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:00:26.872335  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:26.831230 10937 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:26.835697 10814 rpcz_store.cc:276] 0709 13:00:18.280691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.747766 (+467075us) service_pool.cc:224] Handling call
0709 13:00:19.154117 (+406351us) ts_tablet_manager.cc:934] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
0709 13:00:25.046328 (+5892211us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:26.362689 (+1316361us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:26.362821 (+   132us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:26.682339 (+319518us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:26.682652 (+   313us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:26.682959 (+   307us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:26.683021 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":30019,"tablet-copy.queue_time_us":218399,"thread_start_us":147014,"threads_started":2}
I20260709 13:00:26.831453 10936 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 13:00:26.878669  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.880401 10882 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.881179 10189 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.922734 10939 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:26.906773  8712 leader_election.cc:336] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.694s (SENT)
I20260709 13:00:26.906787  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:26.906771  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50946.4us
I20260709 13:00:26.941533  8893 raft_consensus.cc:2376] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5. Re-sending same reply.
I20260709 13:00:26.942441  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:26.943104  8893 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.948747  8748 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:26.908388  8787 connection.cc:256] RPC call timeout handler was delayed by -3.1870759079999402s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
I20260709 13:00:26.902653  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:26.904994  8898 rpcz_store.cc:276] 0709 13:00:21.722186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.352031 (+4629845us) service_pool.cc:224] Handling call
0709 13:00:26.352319 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.389628 (+ 37309us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:26.490815 (+101187us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.711925 (+221110us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.712503 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.712641 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:26.712674 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.863978 (+151304us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b54000488d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:26.864033 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.883261 (+ 19228us) raft_consensus.cc:1685] finished
0709 13:00:26.883342 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.883477 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20575,"prepare.queue_time_us":142961,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":40,"raft.queue_time_us":40059,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":178,"spinlock_wait_cycles":29503232,"thread_start_us":232281,"threads_started":2,"wal-append.queue_time_us":230649}
W20260709 13:00:26.950191  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 47622.9us
I20260709 13:00:26.950906  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.951864  8898 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:26.922904 10752 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:26.910707  8788 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.189s (SENT) [suppressed 45 similar messages]
I20260709 13:00:26.918526 10425 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:00:26.913641  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 119374us
W20260709 13:00:26.914670  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.920077  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:26.883562 10661 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:26.911126 10733 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.994956  8893 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:26.922874 10632 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.925127 10943 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:26.915189 10833 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.917941 10916 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.934329  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.812134  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.952497  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:26.910259 10960 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.956544  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:27.025897  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 183545us
W20260709 13:00:27.026511  8712 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.709s (SENT)
I20260709 13:00:26.935345  9440 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:26.918212 10521 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:00:27.030666  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.974279 10814 tablet_copy_client.cc:806] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
W20260709 13:00:26.958490  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 47827.2us
I20260709 13:00:26.993459 10658 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:26.962002 10936 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:26.950927  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:26.964308  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44330.6us
I20260709 13:00:26.964201  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:27.057859  8889 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.057936  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:26.920001 10958 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.022481  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:27.022686  8896 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.022482  8897 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:26.932876  8891 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
W20260709 13:00:26.921124  8890 rpcz_store.cc:269] 0709 13:00:22.725694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.991652 (+265958us) service_pool.cc:224] Handling call
0709 13:00:26.866373 (+3874721us) spinlock_profiling.cc:243] Waited 3.41 s on lock 0x7b540032b254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:26.866534 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7838365056}
I20260709 13:00:27.063445 10916 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.063874 10916 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.064324 10916 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.065990  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.067229  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.067561 10814 tablet_copy_client.cc:670] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:27.067800  8818 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:27.068681  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.064944  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.070779  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.071842  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.071772 10916 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.072710  8818 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.073526  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.074565  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.075264  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.075646  8818 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:27.076397 10752 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.079732  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.081578 10658 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:27.082427 10943 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.087370  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.052621 10946 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.993956 10878 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.031301  8820 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:26.993242 10661 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.952811  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.104637  8898 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.032308  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.105247  8893 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.058974 10947 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.913152  8894 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:27.047102  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.112224  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:26.998196 10632 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:27.029011  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.057277  8822 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:00:27.058008  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 107203us
I20260709 13:00:27.059042  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:27.114714  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 55765.9us
I20260709 13:00:27.114854  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:27.115646 10632 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.116263  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:27.057018 10572 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:00:27.117769 10756 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.118367 10953 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:27.119015 10940 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.023308  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.059437  8892 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.024560 10945 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:00:27.123960  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.125192  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.125640 10837 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:27.125900  8892 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.126032 10837 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.127187  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:27.126991 10837 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.128131 10837 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.128840  8892 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.128597 10882 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.129987  8895 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.130796  8892 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:27.131721  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.132157  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.030364  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 115315us
W20260709 13:00:27.032783  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 127222us
I20260709 13:00:27.132825  8892 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.133852  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:27.133977  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.106928  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.027240 10955 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:27.135528  8820 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.136489  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.136644  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:27.106973 10977 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.105095 10878 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.053840 10900 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:00:27.156158  8822 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:26.956346 10937 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:00:27.112754  8889 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:27.118193 10572 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:27.118829 10591 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:27.100041  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:27.059949  8712 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.028s (SENT)
I20260709 13:00:27.120478  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.121485 10940 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:27.121930 10911 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.121819  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.118862 10953 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:27.122478 10958 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:27.118206 10756 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:27.123538 10961 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.106758 10916 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:27.137480  8820 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.137511  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:27.182155  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44741.7us
I20260709 13:00:27.182227  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.138478  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:27.183072  8895 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.145916  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:27.183189  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.146958  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.183736  8820 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5. Re-sending same reply.
I20260709 13:00:27.183981  8894 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.147480  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:27.148705  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.184499  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.161556  8892 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.025231 10960 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:27.156756  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.163432  8893 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.163542  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.189941  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:27.190831  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.191147  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:27.191445  8744 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:27.131217 10882 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:27.170697  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 114771us
W20260709 13:00:27.194975  8712 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.030s (SENT)
I20260709 13:00:27.135969  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.170483  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.202358  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.166267  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.146839 10437 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:00:27.166842 10936 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:00:27.205488  8889 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:27.177165  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.208024  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.209103  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.209980  8741 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:00:27.162370 10878 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.210445  8741 pending_rounds.cc:85] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 5
I20260709 13:00:27.210784  8741 pending_rounds.cc:107] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:27.180655  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.211084  8741 raft_consensus.cc:2889] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.211405  8749 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:27.210142  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.212459  8823 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:27.154217 10974 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:27.185034  8820 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:27.193167  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.214059  8746 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.193884 10554 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.194118 10849 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.195737  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.195951  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:27.197130  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.198670  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.198670  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.199375  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:27.173692 10911 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:27.138578 10189 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:00:27.154028 10848 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.147445 10977 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.196341  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:27.210803  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.214622  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.187438 10756 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.173520 10879 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:00:27.218447  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.175525 10903 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:00:27.233903  8712 connection.cc:256] RPC call timeout handler was delayed by -3.4961161699999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
W20260709 13:00:27.234310  8712 connection.cc:256] RPC call timeout handler was delayed by -2.7390291699999807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:00:27.234411  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:27.234637  8712 connection.cc:256] RPC call timeout handler was delayed by -2.6172807629999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.275s for a response
W20260709 13:00:27.235540  8712 connection.cc:256] RPC call timeout handler was delayed by -2.4960291699999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 13:00:27.235594  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:27.235936  8712 connection.cc:256] RPC call timeout handler was delayed by -2.2869807630000309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31170000000000003s for a response
I20260709 13:00:27.220491  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:27.220709  8896 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:27.221913  8894 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:27.221527  8740 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.238341  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:27.222169  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.176880 10856 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:00:27.203277 10839 raft_consensus.cc:2764] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:27.239087  8823 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:27.270320  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.271066  8741 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:27.213596 10974 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.265446  8817 raft_consensus.cc:2368] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:27.220257 10849 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.224247  8892 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:27.224706  8744 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:27.225271  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.244771  8820 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:27.225679  8749 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:27.227473  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.220201  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.255373  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:27.287926  8819 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.256629  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:27.218686 10973 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.203681 10833 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.302127  8821 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.223232  8895 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:27.179932 10658 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.289446  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.269460 10848 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.292748  8820 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:27.297240  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60143.9us
I20260709 13:00:27.318849  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:27.299831  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 117078us
I20260709 13:00:27.324872  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:27.286131 10839 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.285430 10909 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.327397 10189 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:27.300693  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77279.2us
I20260709 13:00:27.311067  8740 raft_consensus.cc:2410] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:27.284835 10849 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.277531 10974 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:27.327771 10189 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:27.329238  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:27.328681  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.333621  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:27.336359  8749 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:27.336609  8740 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:27.337386  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.337484  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.301486 10946 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.338374  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.338857  8749 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.338961  8741 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:27.339656  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.338394  8740 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:27.337416  8748 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:27.345396  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.346136  8748 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:27.349642  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:27.350040 10979 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:00:27.350870  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.353588  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.341341 10189 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:27.371420  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.372455  8740 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 2.
I20260709 13:00:27.374672  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:27.374986  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:27.378988  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.380609  8749 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:27.385310 10903 consensus_queue.cc:1243] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:27.276657 10940 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.390765  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.390919  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.326606 10833 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.391893  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:27.393529  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:27.393602  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:27.330364 10839 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.330508 10909 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:27.446025  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 54237.3us
W20260709 13:00:27.446797  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53355.6us
I20260709 13:00:27.447194  8741 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:27.399192 10189 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:27.416540  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.444094  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46066.8us
W20260709 13:00:27.444905  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51500.1us
I20260709 13:00:27.339706  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.224278 10977 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:27.446468  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67714.5us
I20260709 13:00:27.373021 10973 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.457875  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.396522 10916 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:27.450961  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:27.449096 10986 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:27.390424 10554 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.372212 10950 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.474080  9440 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:27.476111  9441 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.553s	user 0.004s	sys 0.004s
I20260709 13:00:27.449661 10879 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:27.477527 10993 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:00:27.449970  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 131253us
W20260709 13:00:27.514223  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67310.4us
I20260709 13:00:27.514895  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:27.480026  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.449944 10752 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.330868 10990 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 13:00:27.338510 10971 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260709 13:00:27.290426 10953 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.446316 10839 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.510747 10916 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:27.519523  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.516505  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:27.465925 10903 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.511415  9441 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:27.506204 10977 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.528195  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:27.529670 10189 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.542984  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.543758  8749 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:27.464625  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.506139  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:27.520972  8890 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:27.552294 10833 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.500422 10994 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.535854 10947 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.563922  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:27.536576  8748 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.507057 10950 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:27.501751 10847 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:00:27.522666 10846 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.520609 10945 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.520195 10986 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:27.559784  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.508631 10554 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:27.479387 10973 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:27.577642  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.580874  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:27.595460  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:27.604452  9446 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:27.570961 10947 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.603662 10986 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.561458 10964 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.570623 10957 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.549934 10991 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.520287 10848 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.479974 10960 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:27.550310 10958 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:00:27.539513 10752 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:27.638300  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.630486  8746 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:27.573982 10994 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.597016 10846 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.647871  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:27.627662  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:27.649534  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 106880us
I20260709 13:00:27.667394  8749 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.667701  8748 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:27.611486  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:27.651216  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71581.8us
I20260709 13:00:27.644045  8743 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:00:27.681525  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:27.623901 10981 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:00:27.620769 10437 raft_consensus.cc:2764] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:27.619534 10973 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.623548 10947 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.655133 10994 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.623107  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:27.695698  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:27.669550  8748 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.656189 10991 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:27.623112 10964 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:27.683734  8893 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:27.646432 10756 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:27.672142  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.653412 10998 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.699770  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.706205  9447 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:27.655903 10752 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:27.707221  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:27.707955 10756 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:27.708326 10752 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:27.710212 10554 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:27.686302  9441 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.621368 10632 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.615520 10946 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:00:27.667981 10992 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.708771  9447 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 4 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:27.685925 10997 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.655990 10950 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.744725 10950 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:27.745955  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 295202us
I20260709 13:00:27.673821 10960 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:27.682487 10945 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:00:27.703859 10939 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:27.712059 10661 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.596s	user 0.004s	sys 0.006s
I20260709 13:00:27.704088 10964 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.726784 10997 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.794757 10939 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:27.835687 10939 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.836773 10939 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.718852 11005 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.127s
I20260709 13:00:27.834177 10846 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.840062 10846 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.841257 10839 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.842505  9442 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:27.844053 10839 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:27.844342  9441 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:27.793931 10992 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:27.830813 10661 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.847074 10997 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.856451 10661 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:27.857414 10968 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.857944 10968 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:27.859023 10968 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.860574 10968 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.857743  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:27.861393 10985 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:00:27.862823 10995 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:00:27.865309 10632 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:27.866094 10945 consensus_queue.cc:1243] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:27.869527 10991 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:27.851317 10878 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.008s
I20260709 13:00:27.830680 10916 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:27.770062 10849 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.899751 10947 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.891103 10878 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.904206 10878 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:27.904893 10943 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.818s	user 0.006s	sys 0.002s
I20260709 13:00:27.902113 10947 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:27.923813 10837 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.795s	user 0.012s	sys 0.000s
W20260709 13:00:27.944696 10974 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.596s	user 0.004s	sys 0.007s
I20260709 13:00:27.958323 10849 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:27.946592 10992 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.005399 10994 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.015615 10986 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:27.963007 10974 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.047492 10974 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:28.036538 10997 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.044832 10994 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.053910 10997 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.086382  8747 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:27.675384  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.092573  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.094269  8742 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:28.093880  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 424448us
I20260709 13:00:28.095997  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:28.096515  8746 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.097896  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.099306  8741 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:28.099573  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.100418  8746 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.101450  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.102450  8743 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:28.086784  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.103580  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.686646  8744 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:28.094417  8748 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:28.109251  8749 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:28.119177  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.000s
W20260709 13:00:28.119318  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 447601us
W20260709 13:00:28.119817  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.002s	sys 0.000s
W20260709 13:00:28.120484  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 539930us
I20260709 13:00:28.121811  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.130239  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.136700  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.139993  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 342) took 1184 ms. Trace:
W20260709 13:00:28.141258  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 394566us
I20260709 13:00:28.140609  8824 rpcz_store.cc:276] 0709 13:00:26.955477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.957384 (+  1907us) service_pool.cc:224] Handling call
0709 13:00:26.960862 (+  3478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.961187 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.961212 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:27.027487 (+ 66275us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.199921 (+172434us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:27.199958 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.200019 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.139357 (+939338us) raft_consensus.cc:1685] finished
0709 13:00:28.139407 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.139729 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1569,"raft.queue_time_us":1606,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":24730,"thread_start_us":124873,"threads_started":1,"wal-append.queue_time_us":223658}
I20260709 13:00:28.145854  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:28.147611  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:28.150187  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.151677  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1902) took 1064 ms. Trace:
I20260709 13:00:28.152243  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.152153  8816 rpcz_store.cc:276] 0709 13:00:27.087298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.090801 (+  3503us) service_pool.cc:224] Handling call
0709 13:00:28.149614 (+1058813us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:28.151502 (+  1888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32111488}
I20260709 13:00:28.154305  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.155040  8891 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.156219  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:28.156541  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.158512  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1557) took 1002 ms. Trace:
I20260709 13:00:28.158623  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:28.016935 10961 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.004s
I20260709 13:00:28.029544 10973 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.044783 10986 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.197472  8744 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.197821 10572 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:28.164381  8745 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:28.199383 10960 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.199925  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.200778 11006 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.201234  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.201591 11006 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.202436  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.203598  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.204248 10538 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.204674 10538 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:28.206295  9440 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:28.206344  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.206781 10856 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.207257 10856 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:28.209504 10991 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.198993  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.152505  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.169385 10943 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:28.169696  9448 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:00:28.170008  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.172061 10953 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:28.166666  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.006s	sys 0.005s
I20260709 13:00:28.152276  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:28.179662  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.182796 10900 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:28.177917 10947 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.183733  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.184572 10916 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.182647 10658 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.195544  9449 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:28.198403 10964 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.159107  8898 rpcz_store.cc:276] 0709 13:00:27.156392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.202762 (+ 46370us) service_pool.cc:224] Handling call
0709 13:00:27.203041 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:27.204640 (+  1599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.204671 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:27.209710 (+  5039us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.487926 (+278216us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:27.487966 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.488029 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.157967 (+669938us) raft_consensus.cc:1685] finished
0709 13:00:28.158014 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.158306 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1268,"raft.queue_time_us":3784,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8616,"thread_start_us":277850,"threads_started":1,"wal-append.queue_time_us":388569}
I20260709 13:00:28.204275  8745 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:28.215401  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.007s	sys 0.005s
I20260709 13:00:28.211712 10943 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.219127 10943 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.219255 10658 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:28.223095  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.214031 10953 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.216006 10947 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:28.205147 11006 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.214303 10900 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.224361 10953 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.234503  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.209044 10856 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.231441  8740 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:28.237330  8743 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.236531  9441 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:28.237757  8744 raft_consensus.cc:2376] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:28.227495 10960 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:28.240477  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.243386 10973 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:28.245312  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.245182 10961 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.267182 10961 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:28.267516 11004 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.268059 11004 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:28.268724 10961 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.269735  9440 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.245522 10879 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.272188 10879 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.221514 10994 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:00:28.272974 10733 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.265079 10591 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:28.275036 10591 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:28.277144  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:28.279147 10879 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:28.281895 10856 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.246891 10501 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.289740 10501 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.276445 10591 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.291786  8891 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.229368 10900 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.249311  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.252101  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:28.253966  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:28.254262 10947 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.255460 10960 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.225118 10991 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.257318 10964 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:28.255455  9442 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.262879 11002 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.303696 11002 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:28.303771  8745 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:28.305387 11002 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.306185 10988 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:28.263854  9441 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858"
dest_uuid: "112105225cc140459688be5d31dcb071"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:33086
I20260709 13:00:28.206355 10538 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.220952 10658 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:28.299407  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47447us
I20260709 13:00:28.307606  9441 raft_consensus.cc:606] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.288785 11004 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:28.317891 11004 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.318077  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.319006  9441 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.322068  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.323076  9446 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:28.324214  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.327400  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:28.292053 10501 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.328346  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.306762 10988 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:28.335232  8744 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
W20260709 13:00:28.347731  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 195862us
I20260709 13:00:28.347944  8746 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:28.331949 10991 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.357589  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:28.357589  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:28.359434  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.356047  8742 raft_consensus.cc:2376] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6. Re-sending same reply.
I20260709 13:00:28.344973  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:28.362759  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.365648  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.350862 11002 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.366876 10538 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:28.368393  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 216450us
I20260709 13:00:28.373960  9446 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:28.378942 10968 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.518s	user 0.014s	sys 0.005s
I20260709 13:00:28.384147  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.385351  9442 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.391391  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.351132 10911 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.405364 10988 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:28.407348  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:28.412875  9441 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:28.419003  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:28.426343 10911 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.427089  9441 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:00:28.427888  8863 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.765s (SENT)
I20260709 13:00:28.428380  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:28.428835  9441 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.429688  8742 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:28.430306 10998 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.430667  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.431038  9441 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.432818  9441 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.424588  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 556) took 1035 ms. Trace:
W20260709 13:00:28.439992  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.441902  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:28.444077  8746 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:28.458011  9446 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.456915  8743 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:28.459980  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.439987  9448 rpcz_store.cc:276] 0709 13:00:27.382761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.283420 (+900659us) service_pool.cc:224] Handling call
0709 13:00:28.283860 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.284368 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.284396 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.284851 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.349139 (+ 64288us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:28.349421 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.352680 (+  3259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.415767 (+ 63087us) raft_consensus.cc:1685] finished
0709 13:00:28.415850 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.418098 (+  2248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6431,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":63873,"threads_started":1,"wal-append.queue_time_us":108918}
I20260709 13:00:28.473621 10943 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:28.474151  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.474776  8815 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.476214 10943 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:28.476866 10955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.034s	sys 0.064s
W20260709 13:00:28.477458 10955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.034s	sys 0.065s
I20260709 13:00:28.478456  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.479324  8822 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:28.480785 10998 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:28.482126 10916 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:28.482606 10916 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:28.483794 10916 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:28.486938  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 551) took 1116 ms. Trace:
I20260709 13:00:28.487473  9449 rpcz_store.cc:276] 0709 13:00:27.370529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.390548 (+ 20019us) service_pool.cc:224] Handling call
0709 13:00:28.486609 (+1096061us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:28.486844 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652565248}
I20260709 13:00:28.489398  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1559) took 1114 ms. Trace:
I20260709 13:00:28.489800  8740 rpcz_store.cc:276] 0709 13:00:27.374684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.378501 (+  3817us) service_pool.cc:224] Handling call
0709 13:00:28.489165 (+1110664us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b5400200b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:28.489317 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1937634176}
I20260709 13:00:28.497643  8742 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:28.498605  8746 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:28.469849  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 490) took 1096 ms. Trace:
I20260709 13:00:28.499539  9440 rpcz_store.cc:276] 0709 13:00:27.370309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.209340 (+839031us) service_pool.cc:224] Handling call
0709 13:00:28.216333 (+  6993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.270455 (+ 54122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.455010 (+184555us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:28.466877 (+ 11867us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16600448}
I20260709 13:00:28.500492  9443 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:00:28.501845  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.502653  8820 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.502386 10572 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:28.501879  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.484114  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:28.498446  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55547.3us
W20260709 13:00:28.505859  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.476790  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:28.518544  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68300.6us
I20260709 13:00:28.529297  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:28.496665 10900 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:28.544986  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.549809  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.552274  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.528781  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 555) took 1143 ms. Trace:
I20260709 13:00:28.526615 10968 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:28.559749  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.542955 10591 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:28.565237  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:28.566109  8890 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:28.565263 11027 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:00:28.570885  9445 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:28.571586  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 481) took 1245 ms. Trace:
I20260709 13:00:28.571972  9445 rpcz_store.cc:276] 0709 13:00:27.326374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.331341 (+  4967us) service_pool.cc:224] Handling call
0709 13:00:28.483794 (+1152453us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:28.571503 (+ 87709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3924864}
I20260709 13:00:28.562160  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:28.578678  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77019.3us
I20260709 13:00:28.577201  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:28.590651  9448 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.590745  8818 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:28.591110  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 89259.9us
I20260709 13:00:28.577973  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 487) took 1208 ms. Trace:
I20260709 13:00:28.577762  8743 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:28.584271  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:28.606221  9447 rpcz_store.cc:276] 0709 13:00:27.369642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.707990 (+338348us) service_pool.cc:224] Handling call
0709 13:00:27.708275 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:27.708628 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:27.828234 (+119606us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.205159 (+376925us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.575177 (+370018us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b2c000ebf38. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:00:28.575583 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.575846 (+   263us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:28.575885 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.575993 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.577532 (+  1539us) raft_consensus.cc:1685] finished
0709 13:00:28.577644 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.577781 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7196,"raft.queue_time_us":18849,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":27442,"spinlock_wait_cycles":10420096,"thread_start_us":340864,"threads_started":1,"wal-append.queue_time_us":210221}
W20260709 13:00:28.642199  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.587283  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 488) took 1217 ms. Trace:
W20260709 13:00:28.600677  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45082us
I20260709 13:00:28.562426 10591 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:28.648240  9449 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:28.581182  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.590651  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:28.657018  8745 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:28.659265  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:28.661079  8896 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.661173  8895 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:28.662561  9447 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:28.662564  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:28.664208  9446 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:28.665341  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 102729us
I20260709 13:00:28.564934 11021 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.141s
I20260709 13:00:28.666278  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:28.669476 10968 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.672982  8892 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
W20260709 13:00:28.678351  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 326184us
W20260709 13:00:28.665351  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.661619  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.651149 10940 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:28.651280 10591 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.653699 11000 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.691815  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:28.692741  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.653337 10947 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.663314  8898 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:28.697232  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 167334us
I20260709 13:00:28.698330 10425 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.696496 10189 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.700110 10425 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:28.703820  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 122822us
I20260709 13:00:28.704556  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:28.705197 10425 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:28.706058  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.645991  9443 rpcz_store.cc:276] 0709 13:00:27.369874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.862299 (+492425us) service_pool.cc:224] Handling call
0709 13:00:28.500051 (+637752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.501068 (+  1017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.587077 (+ 86009us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54004d1050. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:28.587220 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466059136}
W20260709 13:00:28.710590  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.711047  8815 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:28.712014  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.713644  8822 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:28.701028 11000 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:28.716504  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 125501us
W20260709 13:00:28.720399  8789 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.528s (SENT)
I20260709 13:00:28.720818  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:28.728806 10189 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.713128  8815 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:28.723332 10940 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:28.737313 11029 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:00:28.708614 10953 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:28.850366  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 123905us
I20260709 13:00:28.971629  8749 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.557412  9444 rpcz_store.cc:276] 0709 13:00:27.382472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.278307 (+895835us) service_pool.cc:224] Handling call
0709 13:00:28.278765 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.279372 (+   607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.279402 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.280051 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.367400 (+ 87349us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:28.385087 (+ 17687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.392109 (+  7022us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.510675 (+118566us) raft_consensus.cc:1685] finished
0709 13:00:28.510704 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.526284 (+ 15580us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1533,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":178,"thread_start_us":72533,"threads_started":1,"wal-append.queue_time_us":153422}
I20260709 13:00:28.887825  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:28.794087 10658 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:28.733685 10947 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.734232  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:28.964699  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.970403 11026 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.483s	user 0.004s	sys 0.000s
W20260709 13:00:28.980494  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 254627us
W20260709 13:00:28.972955  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 212590us
W20260709 13:00:28.981860  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.982494 10189 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.985229 11004 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.986487 10953 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:28.987543 11044 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.989449 10900 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:28.991024  8823 raft_consensus.cc:2368] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
W20260709 13:00:28.992326 11038 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.331s	user 0.000s	sys 0.006s
W20260709 13:00:28.993896  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 264024us
I20260709 13:00:28.994129  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1266) took 1796 ms. Trace:
I20260709 13:00:28.996007 10968 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:28.996907 10961 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.034569 10961 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.036979 10961 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:28.998098 11000 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:28.980674 10658 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:29.001374 11043 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.002085 10591 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:29.044499  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.003763  8889 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:29.004522  8818 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:29.006394  8894 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:29.005405  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:29.011368  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:29.045573  8822 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:29.012274  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 343173us
I20260709 13:00:29.012981 10987 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:29.047003  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 326772us
I20260709 13:00:28.965209  8815 raft_consensus.cc:606] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.047358 10987 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.050829  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.051003 10987 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.051051  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:29.051882 11037 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:00:29.041355 10955 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
W20260709 13:00:29.029968  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.032009 10425 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.033972 10968 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.997793 10992 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:29.039963  9442 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.001s
I20260709 13:00:29.040127  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.040581 11044 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:29.043323  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78242.3us
I20260709 13:00:29.026415  8747 rpcz_store.cc:276] 0709 13:00:27.198006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.198110 (+   104us) service_pool.cc:224] Handling call
0709 13:00:27.206589 (+  8479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.087751 (+881162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.993857 (+906106us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54000b6654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:28.994055 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022378880}
I20260709 13:00:29.013871  9446 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:00:29.041558  8741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1561) took 1539 ms (client timeout 1741 ms). Trace:
I20260709 13:00:29.047176 10189 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:29.078115 10189 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:29.078588 10189 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.044010 10658 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:29.077234  8741 rpcz_store.cc:269] 0709 13:00:27.502495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.502635 (+   140us) service_pool.cc:224] Handling call
0709 13:00:29.041334 (+1538699us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:29.041491 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316225408}
I20260709 13:00:29.052493 10909 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.028041 11004 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.092311 10909 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.093415 11004 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:29.044725  9414 leader_election.cc:336] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.046s (SENT)
I20260709 13:00:29.095805  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.055344  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.055523 10940 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.097626  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:29.049467  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.099466  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:29.056697  8789 leader_election.cc:336] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.537s (SENT)
I20260709 13:00:29.101027 10943 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.101523  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.078275 10961 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:00:29.081748  8893 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:29.050316  8815 raft_consensus.cc:614] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:29.102471 10961 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:29.082297  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 461601us
W20260709 13:00:29.085826 11002 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.002s
I20260709 13:00:29.049991 11043 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.087560 10900 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.056623 10916 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.050565 10903 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:00:29.050196 10947 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.050457 10591 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.088090  8889 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.088629  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:29.102674 10953 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:29.057210 10968 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:29.085847 11041 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.093s	user 0.000s	sys 0.003s
I20260709 13:00:29.076877  9442 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:29.101886 10943 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:29.101085  8894 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:29.089649 11044 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.102923 10189 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.120180 10943 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.164078  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1582) took 1030 ms. Trace:
I20260709 13:00:29.055394 10425 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.110250 10953 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:29.170043 10189 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:29.170253  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.120646 10992 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:29.111833 10900 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.168960 10943 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.103279 10937 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.167030  8742 rpcz_store.cc:276] 0709 13:00:28.133796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.430374 (+296578us) service_pool.cc:224] Handling call
0709 13:00:29.163877 (+733503us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b540040c9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:29.164009 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152087808}
I20260709 13:00:29.129611 10947 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:29.168357  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.185941 10847 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.187228 10900 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.188060 10988 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.190423  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.191321 10947 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.191972  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:29.193483 10900 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.193897  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:29.194162 10900 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:29.194587 10900 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.195005 10900 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.174279 10987 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.162211 10538 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:00:29.202595 10538 consensus_queue.cc:1243] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:00:29.167655 11044 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:29.173887 10991 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.204982 10991 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.206599  8892 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:29.207392 10991 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.207551  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.178778 11021 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.181010 11043 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.237829  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:29.244684  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 188111us
I20260709 13:00:29.177417 11012 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
W20260709 13:00:29.269923  8789 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.599s (SENT)
I20260709 13:00:29.236637 11004 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:29.285251  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 245296us
I20260709 13:00:29.190384 10937 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.299130  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.300945  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.292461 11043 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:29.302526  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 95077.6us
I20260709 13:00:29.304986 10425 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.311096 10992 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.329396 11050 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:29.331856 11039 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.346s	user 0.006s	sys 0.002s
I20260709 13:00:29.333820 10856 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.334345 10856 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:29.335390 10856 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.338574 10847 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.342160 10943 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.320127 10987 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.351178 10937 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:29.345218  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44401.9us
I20260709 13:00:29.345316  8817 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:29.352628  8817 pending_rounds.cc:85] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 5
I20260709 13:00:29.352975  8817 pending_rounds.cc:107] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:29.353341  8817 raft_consensus.cc:2889] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:29.348129 10909 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:29.354552  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.354988  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.355072 10909 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.356045 10955 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.355849 10909 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.356585 10955 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.358712 10909 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:29.362026  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 281668us
W20260709 13:00:29.362151  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49027.1us
W20260709 13:00:29.313951  8789 leader_election.cc:336] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.674s (SENT)
W20260709 13:00:29.367324  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 132257us
W20260709 13:00:29.362699  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.304441 11004 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:29.337045 10856 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.244468 10968 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.318337 11058 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.291478 10658 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:29.388706  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 288077us
W20260709 13:00:29.385150  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 358623us
I20260709 13:00:29.288398 10953 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.396477  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:29.398540  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.398466  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.385949 10937 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.400506  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.401491  8894 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:29.401991  8895 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.403134  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:29.403761  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 89900.4us
I20260709 13:00:29.423870  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.426649  8818 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:29.429011  8893 raft_consensus.cc:2368] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:29.431403  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.408461 10968 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.428599  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:29.438326  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 396635us
I20260709 13:00:29.438822 11050 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.440022  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.440691  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:29.440950 11055 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.441644  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.441495 11055 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.442425  8818 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:29.417416  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.445034  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:29.448441  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.449769  8892 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:29.434492 10943 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.454428 11046 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.454994  9442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f"
dest_uuid: "112105225cc140459688be5d31dcb071"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:33086
W20260709 13:00:29.457055  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.456007  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:29.459980  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.457851  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.460637  8818 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:29.434731 11054 raft_consensus.cc:2764] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:29.400574 11058 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.458048  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.466128 11043 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:29.466615 11043 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:29.466996 11043 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.467046 10900 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.468676 10955 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.469709  8893 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:29.470597  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.471648  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.472774  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.473482 10953 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.447074 10937 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:00:29.465090  9442 raft_consensus.cc:606] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.477725  8815 raft_consensus.cc:2410] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:29.483548 10943 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.483675 10961 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.484637  8715 leader_election.cc:400] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:29.484316 10943 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.484316 10961 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:29.484985  8715 leader_election.cc:403] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:29.486645  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:29.488581  8893 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.489200  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.511937 10961 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:29.512267 11051 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.511969 10943 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:29.469231 10900 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.512867 11051 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.515060 11051 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:29.515543 11051 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.515839  8747 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:29.515978 11051 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.517444 11059 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.518033  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.518349  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.518181 11059 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.518765  8891 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:29.520056  8746 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.520306 11059 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:29.518601  8745 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:00:29.521101  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65230.2us
I20260709 13:00:29.522192  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:29.522300  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:29.523583  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.523974 10900 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:29.523111  8746 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:29.511983  8822 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:29.523275  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57536.2us
I20260709 13:00:29.517515  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.465489 10947 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.520231  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:29.532325  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71060.9us
I20260709 13:00:29.487967 11055 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:29.554239  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.554054  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:29.495766 11050 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:29.559854  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.553412  8742 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:29.558947  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.528797 10900 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:29.565522  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:29.568835  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.569763  8890 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:29.551539 10998 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.570935  8817 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:29.553460  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:29.556511 10955 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:29.589051  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77268.9us
I20260709 13:00:29.464941 11004 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.550199 10943 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 13:00:29.714782  8889 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.003s
I20260709 13:00:29.778949  8889 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:29.785079  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.642144  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.589202  8892 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:29.661303  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 129678us
I20260709 13:00:29.788268  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.716424  8647 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:29.788498  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.787441  8742 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:29.789801  8896 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:00:29.588348 11055 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:29.600186  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.660477  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.562615 10988 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:29.646015  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.649107  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:29.826711  8747 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:29.785835  8890 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:29.777936  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 257888us
I20260709 13:00:29.830727  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.716915  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.833884  8748 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:29.789765  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:29.660854  8895 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:29.835461  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.836133  8744 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:29.550524 10961 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:29.650453 11051 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:29.771343 11063 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:00:29.777379 11011 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:29.838904  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46933.3us
I20260709 13:00:29.571612 10900 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.841987  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:29.840729  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.564275 10947 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.853797  8740 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:29.825388 10591 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.771739 10814 tablet_copy_client.cc:538] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:29.828631 11012 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.825765 11000 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.851179  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:29.854804  8892 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:00:29.869501  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 302039us
I20260709 13:00:29.862047  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:29.717137 11043 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:29.844568  8889 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:29.840791 10987 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.852420 10998 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.871320  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:29.884336  8744 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:29.855226 11011 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.838826 11059 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:29.853667 10968 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.891582  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.888026  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:29.895696  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58897.5us
I20260709 13:00:29.905537  8740 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:29.906924  8745 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:29.898236 10916 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:29.908712 11066 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:00:29.909309  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.910858  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:29.916275  8893 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:29.917428  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:29.918157  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60946.2us
I20260709 13:00:29.855186 11071 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:00:29.890987 11011 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.883842 10987 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:29.857354 11005 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.328s
I20260709 13:00:29.922930  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:29.853137 10961 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.955180  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:29.937959  8889 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.892315 10968 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:29.942374  8897 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:29.947124 10988 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:29.875439 11012 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:29.959189  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:29.940964 11043 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:29.964241  8822 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:29.954699  8892 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:29.974284  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48260.2us
I20260709 13:00:29.981127  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:29.978324  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104012us
W20260709 13:00:29.984416  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59351.5us
I20260709 13:00:29.987546  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:29.988511  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65743.5us
I20260709 13:00:29.981945  8896 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:29.949074 11058 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:29.955765 10987 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.961637 11051 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:29.961503 11021 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.004709  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 119209us
W20260709 13:00:29.998224  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 105852us
I20260709 13:00:30.005738  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:29.950191 10916 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:00:29.976310 10988 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:29.984195 11065 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 13:00:29.990974 10992 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.971684 11046 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:00:30.008445  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:30.001490  8647 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.036164  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:30.041003  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:30.001245 11043 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.074870  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:30.032984 10992 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.079785 10992 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.998883 11075 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.088550 11075 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:30.089267 11078 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:00:30.052107 10572 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:30.074918 10856 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.000s
W20260709 13:00:30.106505 11080 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:00:30.027143 11021 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.160427 11075 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:30.161299  8819 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:30.047960 10998 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:00:30.075801 10968 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.161432 11075 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.587668  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 305) took 1072 ms. Trace:
I20260709 13:00:30.076145 11005 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.094002 11089 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:30.587736  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.092320  8647 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:30.375586  8789 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.767s (SENT)
W20260709 13:00:30.420106  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 502955us
I20260709 13:00:30.355036  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 560) took 1217 ms. Trace:
W20260709 13:00:30.512420  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.036s	sys 0.039s
W20260709 13:00:30.528738  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.605542  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.037s	sys 0.039s
W20260709 13:00:30.607118  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 619911us
W20260709 13:00:30.428469 11076 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.390s	user 0.000s	sys 0.000s
I20260709 13:00:30.095889 11046 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:30.550192 11083 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 13:00:30.445945 10953 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.972s	user 0.005s	sys 0.006s
I20260709 13:00:30.608929  9446 rpcz_store.cc:276] 0709 13:00:29.077898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.080400 (+  2502us) service_pool.cc:224] Handling call
0709 13:00:29.091155 (+ 10755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.378809 (+287654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.378839 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.394295 (+ 15456us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.437859 (+ 43564us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:29.437906 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.437975 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.295330 (+857355us) raft_consensus.cc:1685] finished
0709 13:00:30.295359 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.295532 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":286683,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":14135,"thread_start_us":294432,"threads_started":2,"wal-append.queue_time_us":130673}
W20260709 13:00:30.604363  9414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.004s	sys 0.021s
W20260709 13:00:30.627245  9414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.004s	sys 0.021s
I20260709 13:00:30.375262 11085 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.614511  8715 leader_election.cc:400] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
I20260709 13:00:30.636292  8715 leader_election.cc:403] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:30.605743 11005 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:30.616806  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 576153us
I20260709 13:00:30.587597 10916 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:30.627626  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 598105us
W20260709 13:00:30.427659 11086 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.337s	user 0.000s	sys 0.000s
W20260709 13:00:30.635093  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 259658us
I20260709 13:00:30.438263 11051 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:30.635645  8746 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:00:30.639137  9415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.016s	sys 0.031s
W20260709 13:00:30.643338  9415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.016s	sys 0.031s
W20260709 13:00:30.644248  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 613188us
I20260709 13:00:30.274873 10943 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:30.637624  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 590316us
W20260709 13:00:30.608273 11004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.013s	sys 0.016s
I20260709 13:00:30.277022 10572 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:30.423036 10992 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:30.426808 11079 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.337s	user 0.000s	sys 0.000s
I20260709 13:00:30.374607 11071 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.263s
I20260709 13:00:30.614297 11046 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:30.616076 11087 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:30.618558 10658 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:30.642647  8712 leader_election.cc:400] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4.
I20260709 13:00:30.641584 11005 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:30.648496 11004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.013s	sys 0.016s
I20260709 13:00:30.655009  8647 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.424868 11082 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.555s
I20260709 13:00:30.642007 10961 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:30.661188  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.009s	sys 0.041s
I20260709 13:00:30.595054  8743 rpcz_store.cc:276] 0709 13:00:29.514913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.518921 (+  4008us) service_pool.cc:224] Handling call
0709 13:00:29.519229 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.861765 (+342536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.861795 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.892010 (+ 30215us) log.cc:844] Serialized 26 byte log entry
0709 13:00:30.028189 (+136179us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:30.336314 (+308125us) log.cc:844] Serialized 14 byte log entry
0709 13:00:30.582546 (+246232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.582653 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.582707 (+    54us) raft_consensus.cc:1685] finished
0709 13:00:30.582725 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.587610 (+  4885us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45175,"raft.queue_time_us":221351,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":61503,"thread_start_us":666231,"threads_started":3,"wal-append.queue_time_us":188474}
I20260709 13:00:30.644989 11012 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:30.668009  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.041s
W20260709 13:00:30.651342 11047 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.013s	sys 0.036s
I20260709 13:00:30.678242 11071 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:30.678694 11071 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:30.679550 11071 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.682518  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:30.683195  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:30.683792  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:30.684082 10943 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.684721  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:30.684860 10943 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.672796 10658 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.685941 11081 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:00:30.685503 10943 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.685819 10658 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.687331 10814 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 13:00:30.688153 10658 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:30.688515 10943 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:30.689072 10943 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:30.691547 10992 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:30.685051  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:30.691697 10988 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 13:00:30.660666 11089 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.685583  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 303) took 1171 ms. Trace:
W20260709 13:00:30.673491 11047 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.014s	sys 0.036s
I20260709 13:00:30.650782 10916 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.703143  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:30.698689  8741 rpcz_store.cc:276] 0709 13:00:29.514298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.517524 (+  3226us) service_pool.cc:224] Handling call
0709 13:00:29.517804 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.553675 (+ 35871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.553704 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.587730 (+ 34026us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.873869 (+286139us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:29.873909 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.873968 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.685340 (+811372us) raft_consensus.cc:1685] finished
0709 13:00:30.685358 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.685521 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15424,"raft.queue_time_us":196615,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":15873,"thread_start_us":231993,"threads_started":1,"wal-append.queue_time_us":358980}
W20260709 13:00:30.712126  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.006s	sys 0.050s
W20260709 13:00:30.700125 10909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.018s	sys 0.036s
W20260709 13:00:30.714212  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.006s	sys 0.050s
I20260709 13:00:30.722096  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:30.724277  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1589) took 1167 ms. Trace:
I20260709 13:00:30.639385 11085 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.707899 11004 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.724725  8890 rpcz_store.cc:276] 0709 13:00:29.557262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.839988 (+282726us) service_pool.cc:224] Handling call
0709 13:00:29.855831 (+ 15843us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:29.873899 (+ 18068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.873927 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:29.893133 (+ 19206us) log.cc:844] Serialized 52 byte log entry
0709 13:00:29.978067 (+ 84934us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:29.978112 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.978184 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.724045 (+745861us) raft_consensus.cc:1685] finished
0709 13:00:30.724064 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.724220 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5799,"raft.queue_time_us":8398,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":15493,"thread_start_us":35897,"threads_started":1,"wal-append.queue_time_us":101413}
W20260709 13:00:30.739225  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77017.2us
I20260709 13:00:30.707069  8647 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.707269 11090 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.722662 10909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.018s	sys 0.036s
I20260709 13:00:30.741945  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:30.714500 11071 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:30.702768 10988 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.729411 11021 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:30.707391 10992 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:00:30.674412 11051 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.787976  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:30.790668  8898 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:30.742342 10961 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:30.792303  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 177424us
W20260709 13:00:30.789002  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.789487  8647 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:00:30.801875  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:30.802469  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:30.807512  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69376.2us
I20260709 13:00:30.808260  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:30.808889  8893 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:30.811208  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:30.813501 10943 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:30.815215 10943 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.815640 10943 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.816004 10943 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.830849 10961 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:30.831619 10961 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.834599 10961 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:30.835901 11094 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.836494 11094 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.838938 11094 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:30.839494 11094 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.839975 11094 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.843650 11094 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:30.844357 11094 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.844941 11094 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.847301 10961 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:30.848060 10961 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.850564 10961 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:30.801074 10987 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.788s	user 0.005s	sys 0.006s
W20260709 13:00:30.751680 11073 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.017s	sys 0.009s
I20260709 13:00:30.854138  8647 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:00:30.856379  8647 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:30.859226  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 171826us
I20260709 13:00:30.799927 11004 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.867311  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:30.788517 11100 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:30.696992 10572 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:30.802414 11021 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:30.854107 11043 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
W20260709 13:00:30.885725  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 275191us
W20260709 13:00:30.876032  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 220528us
W20260709 13:00:30.874953  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 262550us
W20260709 13:00:30.858646 11073 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.018s	sys 0.009s
I20260709 13:00:30.889322  9446 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:30.852679 10998 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.781136 11096 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:00:30.888126  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:30.871243 10909 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:30.744272 11046 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.780881 11090 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:30.890755 10900 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.018s	user 0.008s	sys 0.003s
I20260709 13:00:30.905941  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:30.904539  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:30.892022  8647 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.829195 10943 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:30.910305  8749 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:30.892529 11087 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:30.913321  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:30.868534 10987 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.911181  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:30.891317 11043 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.906625 11094 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:30.904089 10998 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.904637 11075 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.010114  8745 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:30.890919 11021 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:30.917711  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:30.918320  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:30.906845  9449 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:30.874358 10961 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:00:30.919718  8647 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.917846  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.005735  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:30.897743 10909 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:30.910343 11096 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.018108  8747 raft_consensus.cc:2376] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3. Re-sending same reply.
I20260709 13:00:31.017364  8744 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:31.021727  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.021888  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.022328  8745 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.022460  8749 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.023686  8742 raft_consensus.cc:2376] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:00:31.019364  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:31.028349  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 140416us
I20260709 13:00:31.030006  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:31.015092 11011 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.024s	user 0.009s	sys 0.003s
I20260709 13:00:30.909678  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:31.023737  8748 raft_consensus.cc:2376] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5. Re-sending same reply.
I20260709 13:00:31.040439  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.041736  9449 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:31.041102  8743 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.015252 11046 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.063344  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:31.064132  8816 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:31.064788  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.069485  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.070101  8742 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:30.909510 10900 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.072530  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:30.890515 11100 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.059230  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.023393 11043 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.079385  8747 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 3: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:31.076612  9449 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:31.037632 10916 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.079758  8647 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.083530 10909 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.084645 10909 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.087558  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.088253  8740 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:31.090809  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:31.061666 11047 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.064155  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.091657  8740 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:31.040293 11101 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.092540  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.091835 11043 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.098767 11111 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:31.099350 11111 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:31.069366 11104 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:31.100736  8742 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:31.095225  8647 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.102284  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.077173 11106 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.076074 10988 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.115862  8745 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:31.092728 10937 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.059872 11011 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.102887 11101 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.150422 11117 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.158480  8742 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:00:31.161615  8745 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:31.150593  8647 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to <none>, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:31.165237  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.166635  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.167307  8747 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:31.163627  8647 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 2, term changed from 1 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to <none>, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 4 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:31.110883  9440 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:31.170729  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 254100us
I20260709 13:00:31.172158 11123 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.166196 11117 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.172739 11123 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.172555 11108 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:31.172999  9443 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:31.173177 11123 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.174283 11117 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.174809 11117 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.175242 11095 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:31.175657  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.176270  8742 raft_consensus.cc:2376] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4. Re-sending same reply.
I20260709 13:00:31.177333 11121 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.177913 11121 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.178797 10987 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.179534 10987 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:31.180254 11098 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:00:31.180033 10987 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.147360 11107 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.177155  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.145701 11085 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.186086  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.147274 11094 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:31.213444  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.152343 11101 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.222378  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.166620 10988 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.788308 11083 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:00:31.225687  8789 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.126s (SENT)
I20260709 13:00:31.235577  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.202100  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.233772  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.244716  9443 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:31.175690 11095 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:31.234747  9414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.023s
I20260709 13:00:31.208019  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:31.249965  9414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.023s
I20260709 13:00:31.138753 11047 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.209173 10916 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.175239 11117 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.263685  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:31.265115  8823 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:31.237151 11043 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.252609  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.269625  8742 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:31.252897  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:31.243477 11075 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:31.267041  8789 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.551s (SENT)
I20260709 13:00:31.270864  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.135743 11104 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:31.266587  9443 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:31.277443  8647 catalog_manager.cc:5697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.280010  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:31.131858 11087 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.260298 11004 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:31.286075  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 601713us
I20260709 13:00:31.233794 11094 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.286834  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.260440 11057 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:31.237151 10988 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.158890 11106 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.271762  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.330147  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.330840  8821 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:31.331795  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.331923 11106 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.290560 11100 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:31.291110  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.332912  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:31.292204 11093 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:00:31.285377 11116 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:31.364483  9440 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 4 LEARNER]: Advancing to term 5
I20260709 13:00:31.364912 11116 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:31.365846 11116 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.366884 11116 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.368798  8743 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:31.292984  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.369751  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1284) took 1521 ms. Trace:
I20260709 13:00:31.370136  8817 rpcz_store.cc:276] 0709 13:00:29.848005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.292698 (+1444693us) service_pool.cc:224] Handling call
0709 13:00:31.369691 (+ 76993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:31.371271  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.371908  8817 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:31.372886  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:31.294106 11011 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.295492 11004 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.375182 11004 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.379588  9440 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:31.297824  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.299918 11121 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.243472 11119 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:00:31.264919 10916 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.399072  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:31.301700 11117 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:31.302692  8789 leader_election.cc:336] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.952s (SENT)
I20260709 13:00:31.406467  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:31.407384  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 104776us
W20260709 13:00:31.407795  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0557257610000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 13:00:31.303182 11133 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.409803 11133 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.294128  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.412038  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.413031  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.413419  8817 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.413537  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.414045  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.414616  8741 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:31.415155  8743 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:31.415676  8741 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:31.308159  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:31.416934  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 108911us
I20260709 13:00:31.308789 11112 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:31.309916  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.299934  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.317333  8749 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:31.271910 11021 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.291906  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1286) took 1443 ms. Trace:
I20260709 13:00:31.333762  9447 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:31.333894  9448 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:31.248361 11111 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.423378 10916 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.423918 11096 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.289265 11095 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.334697 11043 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:31.278772 11090 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.346565 11059 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.347143 11057 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:31.437265 11070 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.881s	user 0.010s	sys 0.009s
I20260709 13:00:31.272135 10943 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:31.250689  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 357266us
I20260709 13:00:31.325675  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.362124  8647 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.426178 10591 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.442759 10591 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.443192 10591 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.442693  8647 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.446177  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.447754  8823 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:31.448779  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.444456  8647 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 3 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:00:31.380616 11005 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.000s
I20260709 13:00:31.380478 11101 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.296133 11094 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.214732 11107 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:31.398406  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 321904us
W20260709 13:00:31.405727 11089 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.000s
I20260709 13:00:31.466285 11089 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5.
I20260709 13:00:31.285732 10998 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:31.409173  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:31.468240  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59165.9us
W20260709 13:00:31.420120  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 133420us
I20260709 13:00:31.471079  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.472010 10903 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.421751 10987 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.421907  8743 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:31.422426  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:31.475395  8647 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:31.426689  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.420087  8819 rpcz_store.cc:276] 0709 13:00:29.848615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.851194 (+  2579us) service_pool.cc:224] Handling call
0709 13:00:29.863575 (+ 12381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:30.582386 (+718811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.291653 (+709267us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540040a454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:31.291842 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320061824}
I20260709 13:00:31.384753 10900 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:31.431082  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.424957 11095 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.432312 11112 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:31.433879 10988 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.434489 11043 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:31.434839  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.435635 11047 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.428541 11106 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.430838 11059 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.457489  8817 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:00:31.457930  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.489697  8647 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:31.506990  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.507890  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.459538  9446 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:31.459856  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.508610 10987 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:00:31.509001  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49250.6us
I20260709 13:00:31.465412  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.478731  8821 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:31.510075 10937 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.510649 10937 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.510951  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.511569  8821 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:31.512558  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:31.513260 11012 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.513659 11012 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.513981 11012 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:31.516484 11069 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.652s	user 0.010s	sys 0.013s
I20260709 13:00:31.519642  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.449405  9445 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:31.487592  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70066.9us
I20260709 13:00:31.483702  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:31.489777  8819 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:31.490869  8815 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:31.466321  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.492821 10947 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.485947 11087 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.498935  8820 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:31.497934 11094 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.472951 10903 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:00:31.521378  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:31.523772 11101 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:31.473482 11075 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:00:31.486737 10998 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.631145 11121 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.528576  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:00:31.530179 10988 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.530944  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.631995 11121 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:31.633160  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.531598  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1892) took 2073 ms. Trace:
W20260709 13:00:31.485750 10994 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.634s	user 0.023s	sys 0.034s
I20260709 13:00:31.633876  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:31.634135 10994 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.795s	user 0.023s	sys 0.034s
I20260709 13:00:31.635349 10994 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:31.635793 10994 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:31.599785 11021 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.636566 10994 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.539068  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.636935 11021 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.638244  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.638967 11021 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:31.639459  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.640723  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.633904  8894 rpcz_store.cc:276] 0709 13:00:29.458052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.461062 (+  3010us) service_pool.cc:224] Handling call
0709 13:00:29.477585 (+ 16523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.477989 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.478017 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.478512 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:00:29.587172 (+108660us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:29.587226 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.587328 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:31.515868 (+1928540us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:31.521085 (+  5217us) raft_consensus.cc:1685] finished
0709 13:00:31.521107 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.531507 (+ 10400us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9105,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":16941,"spinlock_wait_cycles":119358720,"thread_start_us":108246,"threads_started":1,"wal-append.queue_time_us":364382}
I20260709 13:00:31.643563  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.520556  8647 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 2, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.541915  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.646387  8748 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:31.505336 11071 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.760s	user 0.011s	sys 0.000s
I20260709 13:00:31.545979  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:31.498584 11096 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:31.551323  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.552006  8749 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:00:31.519326 11059 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.457233 11005 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.567986 11046 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.552348  9442 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:31.581637 11112 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.721771 11123 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.722349 11012 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.582299  8896 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:31.582458  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.532178 11106 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.724133  8896 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:31.724252 11123 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:31.543220 11004 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.725544 11004 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.725968 11004 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.585656  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:31.586814  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:31.728111  8890 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.592214  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.591184  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.552610 10538 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.729020 10538 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.729058  8895 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:31.594954  8898 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:31.730367  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.730844  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.730893  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.731812  8898 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:31.732671  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:31.733728  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.734303  8816 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:31.568393  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1283) took 1705 ms. Trace:
I20260709 13:00:31.734992  8815 rpcz_store.cc:276] 0709 13:00:29.847646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.434575 (+1586929us) service_pool.cc:224] Handling call
0709 13:00:31.552458 (+117883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:31.591817  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.735944  8819 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 2.
W20260709 13:00:31.736760 11138 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.091s	user 0.001s	sys 0.001s
W20260709 13:00:31.600124  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 121138us
I20260709 13:00:31.741142  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:31.742255  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.742866  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.743290  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:31.743871  9443 raft_consensus.cc:2376] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5. Re-sending same reply.
I20260709 13:00:31.581714 10903 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:31.631373 11107 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.744933 10903 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:31.745199 11107 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:31.745443 11121 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.745844 10903 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.746349 11121 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.746627 11107 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.730942  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:31.747093 11097 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:31.747876  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.747967  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:31.650436  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:31.748962  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.749049  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:31.749639  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 99325.7us
I20260709 13:00:31.750070  8898 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:31.750241  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.750780  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:31.527956  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.751920  8898 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:31.752285  9414 connection.cc:256] RPC call timeout handler was delayed by -1.1133122930000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24150000000000002s for a response
I20260709 13:00:31.573020 11094 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:31.657996  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 136746us
I20260709 13:00:31.568884 11087 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.658986  8647 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 3, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:31.679283  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.697909 11140 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.596797 10591 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.665374 11096 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.705883  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:31.707300 11141 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:00:31.711485 11059 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.698830 11090 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.725005  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.729702 10538 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.731717  9445 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:31.510039 10987 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:31.756762  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 112362us
I20260709 13:00:31.777168  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:31.749119  9449 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:31.777660  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 242309us
I20260709 13:00:31.757320  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.761431  8748 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:31.769968  8890 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:31.770800  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.771068  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.779896  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:31.724951  8896 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.772712  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:31.766682  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 174634us
I20260709 13:00:31.750797  8895 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:31.788563  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:31.793150  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.794238  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.598631 11043 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:31.660893 11021 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 13:00:31.798375  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.799695  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.765630 10988 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.800171  8897 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:31.773715 11107 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:31.771942 11096 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.799898  9448 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:31.805863  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:31.771569 10900 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.806519  8748 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.805975  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.808290  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:31.810110  9449 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:31.777405 11090 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.774957 11073 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.807888  9441 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:31.808466 11012 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:31.762392 11046 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.831573 11073 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.805200 10988 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.833137  9448 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.775238 10916 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:31.809202 11118 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.801707 11116 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.878316  8824 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:00:31.878815  8824 pending_rounds.cc:85] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 6
I20260709 13:00:31.879071  8824 pending_rounds.cc:107] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:00:31.879340  8824 raft_consensus.cc:2889] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:31.808857 11119 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.771700 11047 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:00:31.843128  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.819746  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:31.809060 11011 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:31.885093  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103435us
I20260709 13:00:31.893344  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.807052 11121 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.775993 11133 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:00:31.849670  8818 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:31.852968  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.833582  9445 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.786947 10998 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:00:31.851840 10916 consensus_queue.cc:1243] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:31.819470 11090 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.808347 11107 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:31.881297  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 110344us
I20260709 13:00:31.840214  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.912362  8816 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:31.796216 11085 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:31.885915  8894 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:31.790340 11140 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.813512 11146 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
W20260709 13:00:31.891399  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48403.6us
I20260709 13:00:31.929088  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.801887 11100 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.930788 11100 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:31.931391 11100 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.813679 11089 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.846971 10988 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.846477 10909 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.941836 10909 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:31.900308  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90956.5us
I20260709 13:00:31.944821  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:31.945802  8819 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:31.947623  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 166132us
I20260709 13:00:31.896881 11011 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.908643 11075 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:31.889593  9443 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:31.895628 11012 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.948796  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.906776  8748 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:31.899871 10425 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:31.896692  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.948669  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.896050 11119 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:31.876750 10847 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:00:31.846249 10937 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:31.913563  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:31.988541  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:31.972800  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:31.977974  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.989647  8893 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:31.979664  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50731.4us
I20260709 13:00:31.990828  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:31.907900 10572 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:00:31.964643 11011 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:31.993638  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45101.1us
I20260709 13:00:31.975124 11075 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.004575  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:31.980634 11121 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.996351  8748 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:31.998857  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:31.848337 11073 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:32.022877  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78201.8us
I20260709 13:00:32.023470  9447 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:32.023839  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:32.005452  8892 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.993867  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.022270  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:32.029415  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:32.034715  8895 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:31.995386 11123 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:31.997602 10572 consensus_queue.cc:1243] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:32.044379  8816 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:32.044996  8823 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:31.996031 10425 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:32.008980 11075 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:32.036700  8815 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:32.069775  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.068543  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.077235 11154 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.034605 11097 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:32.079857  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:32.083052  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.083110  8895 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.043980 10591 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:32.069053 11089 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:32.090183  8862 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.090s (SENT) [suppressed 32 similar messages]
I20260709 13:00:32.098424  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.069669 11155 raft_consensus.cc:2764] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:32.079831 11094 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.109037  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:32.079639 11123 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:32.095880 10658 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:32.120462  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:32.097311 11085 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:32.126142  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 103998us
I20260709 13:00:32.161468  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:32.115025 11046 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.107987 11089 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.107564 11059 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.096200 11097 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:32.104856  8893 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 3: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:32.043597 11107 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:32.163831  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.169579  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:32.170051  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 258624us
I20260709 13:00:32.150887 11116 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:32.181463  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 150945us
I20260709 13:00:32.181756  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.186277  8893 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:32.188516  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:32.200260  8898 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:32.206651  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.174928 11089 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.172632 11146 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:32.210959  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:32.215117  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:32.211347  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.216432  8898 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:32.222349  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.093914 11119 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.176592 11172 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:00:32.258714  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.259438  8893 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:32.179745 11170 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:32.248999 11154 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.207001 11107 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.253381 11140 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.168861 10425 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.282723  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.276916  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.254136 10937 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:32.292152  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.293139  8889 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:32.287022  8893 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:32.256932 11173 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.217757 11155 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:00:32.280035 11146 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:32.264643 10900 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:32.267827 11154 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.276508 10658 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:32.325067  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.277011 11046 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:32.272071 11073 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.208067 11165 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:00:32.218062 11123 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.278306 11158 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:32.303062 11124 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.125s	user 0.003s	sys 0.005s
I20260709 13:00:32.285250 11097 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.377960  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:32.436311  8898 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:32.318801 11173 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:32.467152  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 346764us
W20260709 13:00:32.336086 11164 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:00:32.469535  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.334679 11140 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:32.454799 11156 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:00:32.318147 11170 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:32.293578 11174 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:32.335522 11162 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:00:32.353259 11004 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.004s
W20260709 13:00:32.335628 11152 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 13:00:32.329221 10538 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 13:00:32.472275  8897 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:00:32.514706  8897 pending_rounds.cc:85] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 6
I20260709 13:00:32.515000  8897 pending_rounds.cc:107] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:00:32.515254  8897 raft_consensus.cc:2889] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:32.329573 11148 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:00:32.462008 11100 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.005s
W20260709 13:00:32.424626 11096 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
W20260709 13:00:32.460462  9444 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.698s	user 0.007s	sys 0.004s
I20260709 13:00:32.333498 11166 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.505321  8898 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:32.530104  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47585.7us
I20260709 13:00:32.466485 11158 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:32.506362  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.501927  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:32.356809 11146 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.484254  8741 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:32.535362  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:32.536123  8741 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:32.536921  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.538136  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:32.538702  8741 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.453687 10900 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.322013 11116 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.484519 11173 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.498006 11094 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.454555 11011 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.348687 10988 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.371227 11046 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.501466 11184 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.528115  8897 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:32.531100  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:32.531262  8749 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.531363  8748 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:32.358510 11154 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.325901 10658 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.507313 11140 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.530102 11166 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.531023  9444 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:32.516877 11004 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.516764 11089 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:32.567035  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 308) took 3011 ms (client timeout 2415 ms). Trace:
W20260709 13:00:32.569008  8746 rpcz_store.cc:269] 0709 13:00:29.550229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.551019 (+   790us) service_pool.cc:224] Handling call
0709 13:00:32.560864 (+3009845us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400463cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:32.561013 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2455523200}
I20260709 13:00:32.518829 10538 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:32.564986 11090 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.004s
I20260709 13:00:32.531293 11100 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.597941  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 498) took 1559 ms. Trace:
I20260709 13:00:32.598510  9446 rpcz_store.cc:276] 0709 13:00:31.038560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.039763 (+  1203us) service_pool.cc:224] Handling call
0709 13:00:31.068936 (+ 29173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.513403 (+444467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.597163 (+1083760us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:32.597861 (+   698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898288000}
W20260709 13:00:32.574931  8892 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
I20260709 13:00:32.626958  8821 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:32.567849 11166 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.684433  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.625663 10937 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.593873 11170 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.653352 11104 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.610016 11004 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.596469  8745 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:32.645968  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 321) took 1108 ms. Trace:
I20260709 13:00:32.593453 11185 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:00:32.652582 11146 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.612015  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:32.610620 11043 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:32.676573  8892 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:32.609724 11189 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:32.561558 11075 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.608930 11195 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.601405  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.711336  8746 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:32.651342  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1606) took 1064 ms. Trace:
I20260709 13:00:32.559540 11174 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:32.671818  8749 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:32.600929 11090 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:32.672008  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110038us
I20260709 13:00:32.531277 11096 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:32.653497 11088 consensus_queue.cc:1175] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7fd110c312c14e17a2bbe6d453493cf2" responder_term: 5 status { last_received { term: 5 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 5 index: 4 } }
I20260709 13:00:32.700232  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 320) took 1163 ms. Trace:
W20260709 13:00:32.700722  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88928.3us
I20260709 13:00:32.672279 11059 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:32.694439  8744 rpcz_store.cc:276] 0709 13:00:31.537197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.546487 (+  9290us) service_pool.cc:224] Handling call
0709 13:00:31.737943 (+191456us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:31.738117 (+   174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.775883 (+ 37766us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:31.970422 (+194539us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.467003 (+496581us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.496777 (+ 29774us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.496922 (+   145us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:32.496960 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.497032 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.645650 (+148618us) raft_consensus.cc:1685] finished
0709 13:00:32.645755 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.645901 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44403,"prepare.queue_time_us":102856,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":37611,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":37064,"spinlock_wait_cycles":3074304,"thread_start_us":580889,"threads_started":2,"wal-append.queue_time_us":579121}
I20260709 13:00:32.712237  8894 rpcz_store.cc:276] 0709 13:00:31.586581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.643313 (+ 56732us) service_pool.cc:224] Handling call
0709 13:00:32.651177 (+1007864us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:32.651282 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556245376}
I20260709 13:00:32.715823  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:32.722350  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:32.719583  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:32.726320  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:32.718144  8740 rpcz_store.cc:276] 0709 13:00:31.536874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.537691 (+   817us) service_pool.cc:224] Handling call
0709 13:00:31.551117 (+ 13426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:31.592552 (+ 41435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.592580 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:31.627709 (+ 35129us) log.cc:844] Serialized 52 byte log entry
0709 13:00:31.882719 (+255010us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:31.883006 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:00:31.897969 (+ 14963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.898048 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.700010 (+801962us) raft_consensus.cc:1685] finished
0709 13:00:32.700030 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.700172 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9400,"prepare.queue_time_us":90646,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":35,"raft.queue_time_us":113436,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":47224,"thread_start_us":116764,"threads_started":1,"wal-append.queue_time_us":274361}
I20260709 13:00:32.728931  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.627707 11158 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.697559 10937 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:32.730314  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.818480  8741 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:32.757211  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:32.709934 11043 consensus_queue.cc:1243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:00:32.701251 11198 raft_consensus.cc:2764] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:00:32.851451  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125302us
I20260709 13:00:32.763154  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.730086  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:32.877972  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.851094  8744 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.000828  8741 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:33.002195  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:33.002863  8892 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:32.905143  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.004463 11118 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.009871  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:33.010934  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:33.011909  8741 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.012859  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:32.819778 10814 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:00:33.015355  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:32.825486  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 501) took 1718 ms (client timeout 2033 ms). Trace:
I20260709 13:00:32.842059 11004 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.019306 10937 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.019937 11100 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.019976  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1618) took 1250 ms. Trace:
I20260709 13:00:33.000283 11146 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.022456  8892 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:00:32.818867 10988 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.822371 11177 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:32.820485 10947 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:32.821868 11096 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.834514 11075 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.881520 11046 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:33.002261 11140 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.033413  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:33.023391  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:33.053704  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:33.037600  8898 rpcz_store.cc:276] 0709 13:00:31.770023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.548236 (+778213us) service_pool.cc:224] Handling call
0709 13:00:32.563132 (+ 14896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.565002 (+  1870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.565046 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.565680 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.683184 (+117504us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:32.683237 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.683313 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.019769 (+336456us) raft_consensus.cc:1685] finished
0709 13:00:33.019793 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.019917 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1454,"raft.queue_time_us":86190,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":117070,"threads_started":1,"wal-append.queue_time_us":202906}
I20260709 13:00:32.822125 11193 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.882400 11090 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.835752 11106 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:32.792912 11011 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:32.721525 11174 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:32.728371 11059 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.067171 11059 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.010803 11089 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.046419  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:32.849134 11160 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:33.014678 11158 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.016907 11095 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.068230 11160 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.068948 11095 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.010097 11187 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:33.016608 11118 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.068221  8740 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3.
I20260709 13:00:33.072669 11187 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.073225  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:33.074045 11118 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:33.074586  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:33.075924  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:33.115020  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:32.829456 11194 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.318s
I20260709 13:00:33.124049  8898 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:33.075827 11187 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.064090 11185 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:33.135437  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 123711us
I20260709 13:00:33.137347  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1615) took 1506 ms. Trace:
I20260709 13:00:33.047667 11004 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:33.147984 11202 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:00:33.137766  8889 rpcz_store.cc:276] 0709 13:00:31.630467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.326058 (+695591us) service_pool.cc:224] Handling call
0709 13:00:32.354912 (+ 28854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.355224 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:32.672293 (+317069us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.136725 (+464432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.136781 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.136832 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.136987 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.137278 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54189,"prepare.queue_time_us":396798,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":24698,"spinlock_wait_cycles":4224,"thread_start_us":777553,"threads_started":2}
W20260709 13:00:33.168187  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.048497 11146 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:33.021257  9442 rpcz_store.cc:269] 0709 13:00:31.106768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.131194 (+ 24426us) service_pool.cc:224] Handling call
0709 13:00:32.825220 (+1694026us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54005b7554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:32.825415 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562107648}
I20260709 13:00:33.053515 10947 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:33.175869  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 364) took 1990 ms (client timeout 1977 ms). Trace:
I20260709 13:00:33.176177 11174 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:33.177102  8889 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:00:33.210649  8820 rpcz_store.cc:269] 0709 13:00:31.185624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.185769 (+   145us) service_pool.cc:224] Handling call
0709 13:00:33.175640 (+1989871us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400076f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:33.175784 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660359936}
I20260709 13:00:33.228917 11004 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.236056 10947 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.138747 11180 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:33.181254 11205 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:00:33.250205  8740 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:33.150426 11185 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.262252 11185 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.190462 11095 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.278647  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:33.286960  8897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.000s	sys 0.021s
W20260709 13:00:33.289618  8897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.000s	sys 0.021s
I20260709 13:00:33.290023  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:33.291896  8820 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:33.293110  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1621) took 1265 ms. Trace:
I20260709 13:00:33.255734 11180 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:33.293444 11166 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.005s
W20260709 13:00:33.293166 11120 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.875s	user 0.003s	sys 0.000s
I20260709 13:00:33.303745  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.293603  8897 rpcz_store.cc:276] 0709 13:00:32.028137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.557174 (+529037us) service_pool.cc:224] Handling call
0709 13:00:32.563822 (+  6648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.564055 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:32.680085 (+116030us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.291618 (+611533us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.292874 (+  1256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:33.292924 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.292953 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.292995 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.293066 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1047,"prepare.queue_time_us":288214,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":6465,"thread_start_us":693136,"threads_started":2}
I20260709 13:00:33.305032  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1907) took 1673 ms. Trace:
I20260709 13:00:33.305610  8895 rpcz_store.cc:276] 0709 13:00:31.631572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.464924 (+833352us) service_pool.cc:224] Handling call
0709 13:00:32.465250 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.594354 (+129104us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:00:32.594390 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.630430 (+ 36040us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.724871 (+ 94441us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:32.724928 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.725026 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.180683 (+455657us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:33.301619 (+120936us) raft_consensus.cc:1685] finished
0709 13:00:33.301634 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.304954 (+  3320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72923,"raft.queue_time_us":88895,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7133,"spinlock_wait_cycles":21559040,"thread_start_us":100628,"threads_started":2,"wal-append.queue_time_us":342688}
I20260709 13:00:33.307590  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1289) took 2453 ms. Trace:
I20260709 13:00:33.308068  8822 rpcz_store.cc:276] 0709 13:00:30.854226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.869102 (+ 14876us) service_pool.cc:224] Handling call
0709 13:00:30.869553 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:30.870012 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:30.870052 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:30.886109 (+ 16057us) log.cc:844] Serialized 52 byte log entry
0709 13:00:31.067411 (+181302us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:31.067454 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.067528 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.051677 (+1984149us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:33.307293 (+255616us) raft_consensus.cc:1685] finished
0709 13:00:33.307311 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.307512 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151888,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":33599,"spinlock_wait_cycles":53527808,"thread_start_us":156487,"threads_started":1,"wal-append.queue_time_us":268459}
I20260709 13:00:33.311882  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:33.312426  8822 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.314312 11146 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.319984  8744 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:33.320444  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:33.320899  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 369) took 1056 ms. Trace:
I20260709 13:00:33.321198  8822 rpcz_store.cc:276] 0709 13:00:32.264894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.311632 (+1046738us) service_pool.cc:224] Handling call
0709 13:00:33.320850 (+  9218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:33.323920  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:33.324574  8744 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:33.326637  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.327661 10658 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.329347 10658 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.331437 10658 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:33.331907  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 318) took 1795 ms (client timeout 2091 ms). Trace:
I20260709 13:00:33.333112  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.333974  8820 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:33.332332  8745 rpcz_store.cc:269] 0709 13:00:31.536209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.538735 (+  2526us) service_pool.cc:224] Handling call
0709 13:00:33.330532 (+1791797us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54004655d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:33.331846 (+  1314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206238080}
I20260709 13:00:33.319365 11166 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.340235  8749 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:33.339926  8745 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:33.345623 11057 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:33.348976 11173 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.000s
I20260709 13:00:33.347991  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.349429 11173 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.349835  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:33.352656  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:33.355315 11146 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.355618 11173 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.355753 11146 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.356168 11146 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:33.356634 11154 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.002s
I20260709 13:00:33.357048 11154 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.358206  8746 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:33.354998  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:33.359272  8822 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.359138  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.368790  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:33.371932 10987 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:33.372480 10987 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:33.373543 10987 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:33.357134 11005 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:33.389174  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.033146  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1618) took 1556 ms. Trace:
I20260709 13:00:33.694888  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1299) took 1801 ms. Trace:
W20260709 13:00:34.049803  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 681407us
I20260709 13:00:34.055611  8816 rpcz_store.cc:276] 0709 13:00:31.893224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.994735 (+101511us) service_pool.cc:224] Handling call
0709 13:00:31.995037 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.071402 (+ 76365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.694614 (+1623212us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:33.694815 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113066880}
I20260709 13:00:33.696990  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:33.411950  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.049355  8896 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.690826  8823 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:33.408327  8743 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:00:33.385486  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1609) took 1764 ms (client timeout 2209 ms). Trace:
W20260709 13:00:33.844480  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.058918  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 647365us
W20260709 13:00:33.431191  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.062516  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:33.830298  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1920) took 2558 ms (client timeout 2115 ms). Trace:
I20260709 13:00:33.441948  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1613) took 1812 ms. Trace:
I20260709 13:00:34.054279  8749 rpcz_store.cc:276] 0709 13:00:32.477113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.885972 (+408859us) service_pool.cc:224] Handling call
0709 13:00:34.032889 (+1146917us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54001c8254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.033072 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751518976}
I20260709 13:00:33.386425  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:34.061793  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.816962  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1293) took 1979 ms. Trace:
I20260709 13:00:34.064543  8819 rpcz_store.cc:276] 0709 13:00:31.837483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.837810 (+   327us) service_pool.cc:224] Handling call
0709 13:00:31.848956 (+ 11146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.946328 (+ 97372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.816669 (+1870341us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b540001f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:33.816880 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222282496}
I20260709 13:00:33.390340  8822 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:34.056839  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.065670  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 377) took 1377 ms. Trace:
W20260709 13:00:34.065816  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.066047  8822 rpcz_store.cc:276] 0709 13:00:32.688220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.354692 (+666472us) service_pool.cc:224] Handling call
0709 13:00:34.065616 (+710924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.066949  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.067646  8822 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:33.690392  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1932) took 2103 ms (client timeout 1674 ms). Trace:
W20260709 13:00:34.063292  8817 rpcz_store.cc:269] 0709 13:00:31.257796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.372680 (+114884us) service_pool.cc:224] Handling call
0709 13:00:33.815651 (+2442971us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54003e30d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:33.815839 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64033664}
I20260709 13:00:34.075412  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.076009  8816 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:34.076695  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.077237  8816 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:34.058696  8891 rpcz_store.cc:269] 0709 13:00:31.621533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.034411 (+412878us) service_pool.cc:224] Handling call
0709 13:00:33.385246 (+1350835us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000483d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:33.385403 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2779067136}
I20260709 13:00:34.063326  8893 rpcz_store.cc:276] 0709 13:00:31.629938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.322422 (+692484us) service_pool.cc:224] Handling call
0709 13:00:32.325359 (+  2937us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:32.544624 (+219265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.544657 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:32.550154 (+  5497us) log.cc:844] Serialized 52 byte log entry
0709 13:00:32.680949 (+130795us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:33.010355 (+329406us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.014170 (+  3815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.308585 (+294415us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18003807a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:33.441614 (+133029us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:33.441669 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.441714 (+    45us) raft_consensus.cc:1685] finished
0709 13:00:33.441745 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.441886 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30425,"prepare.queue_time_us":205050,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":65717,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":24047,"spinlock_wait_cycles":474498688,"thread_start_us":391746,"threads_started":2,"wal-append.queue_time_us":441980}
I20260709 13:00:34.064008  8747 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:00:34.080340  8713 connection.cc:256] RPC call timeout handler was delayed by -1.4580864559999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 13:00:34.080858  8713 connection.cc:256] RPC call timeout handler was delayed by -1.3617864560000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 13:00:34.068673  8821 rpcz_store.cc:269] 0709 13:00:31.586954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.632881 (+ 45927us) service_pool.cc:224] Handling call
0709 13:00:33.690044 (+2057163us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b5400075b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:33.690294 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2284838144}
I20260709 13:00:34.085438  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.091939  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 319) took 2542 ms. Trace:
I20260709 13:00:34.092432  8743 rpcz_store.cc:276] 0709 13:00:31.536543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.546707 (+ 10164us) service_pool.cc:224] Handling call
0709 13:00:33.407887 (+1861180us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:00:33.408088 (+   201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.061954 (+653866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.078948 (+ 16994us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2926976}
W20260709 13:00:33.409938 11107 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.125s	user 0.010s	sys 0.002s
W20260709 13:00:33.430133 11210 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.095s	user 0.000s	sys 0.003s
I20260709 13:00:33.695897 10994 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.134996  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:34.135769  8742 raft_consensus.cc:2368] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8.
I20260709 13:00:34.136641  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:34.137904  8817 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:00:34.138532  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 372) took 1806 ms. Trace:
I20260709 13:00:34.138829  8817 rpcz_store.cc:276] 0709 13:00:32.332518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.099421 (+1766903us) service_pool.cc:224] Handling call
0709 13:00:34.137705 (+ 38284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.138291 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.138481 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.139763  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.140246  8817 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.141530  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:34.142088  8743 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.611650 11087 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:33.686894 11154 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.084578 10987 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.146066 11160 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:33.413089 11193 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.147101  8892 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:34.147159  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:34.147859  8747 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:34.148011  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1623) took 1669 ms. Trace:
I20260709 13:00:34.149506  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 388) took 1095 ms. Trace:
I20260709 13:00:34.149855  8892 rpcz_store.cc:276] 0709 13:00:33.054596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.302081 (+247485us) service_pool.cc:224] Handling call
0709 13:00:33.302787 (+   706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.147562 (+844775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.149271 (+  1709us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540046df54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:34.149452 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942279040}
I20260709 13:00:34.153484 11107 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.154135 11107 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.155503  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1622) took 1677 ms. Trace:
W20260709 13:00:34.155817  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.155859  8745 rpcz_store.cc:276] 0709 13:00:32.478344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.341549 (+863205us) service_pool.cc:224] Handling call
0709 13:00:33.342129 (+   580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.342337 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.342373 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:33.344497 (+  2124us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.628756 (+284259us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:33.628812 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.628920 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.155313 (+526393us) raft_consensus.cc:1685] finished
0709 13:00:34.155331 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.155452 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":281684,"threads_started":1,"wal-append.queue_time_us":806917}
I20260709 13:00:34.156647 11107 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.074961 10538 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.100291  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:34.119642 11112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.396s	user 0.007s	sys 0.021s
W20260709 13:00:34.157794 11112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.434s	user 0.007s	sys 0.021s
I20260709 13:00:34.157857  8749 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:34.079707 11021 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.159230 11021 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:33.830750  8815 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:34.161545 10814 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:00:33.402920 11173 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:33.479471 11178 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:34.015645 11214 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.686s	user 0.002s	sys 0.000s
I20260709 13:00:33.461901 11172 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.405558 10947 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.692464 11043 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:33.636525 11012 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:33.487673 10943 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:33.408481 11166 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.130031  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.197921  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.198294  8822 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:34.044119 11054 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.582522 11118 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.148427  8744 rpcz_store.cc:276] 0709 13:00:32.478645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.342653 (+864008us) service_pool.cc:224] Handling call
0709 13:00:33.353883 (+ 11230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.354198 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.354224 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:33.365516 (+ 11292us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.473968 (+108452us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.525978 (+ 52010us) log.cc:844] Serialized 14 byte log entry
0709 13:00:33.526254 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.526349 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.147824 (+621475us) raft_consensus.cc:1685] finished
0709 13:00:34.147844 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.147960 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51711,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":1169,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":4090,"spinlock_wait_cycles":1274240,"thread_start_us":108029,"threads_started":1,"wal-append.queue_time_us":706693}
I20260709 13:00:34.164635  8742 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:34.206195  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.207011  8893 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.148803  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:34.207701  8747 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:34.165983  8745 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:34.099026  8821 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:34.172765  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:34.209333  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 371) took 1877 ms. Trace:
I20260709 13:00:34.031697 11108 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:33.408854 11117 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:34.211376  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1625) took 1698 ms. Trace:
I20260709 13:00:34.163254  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:34.211911  8745 rpcz_store.cc:276] 0709 13:00:32.512737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.157210 (+1644473us) service_pool.cc:224] Handling call
0709 13:00:34.165729 (+  8519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.208675 (+ 42946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.211324 (+  2649us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:34.091574 11146 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
W20260709 13:00:34.212592  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49434.7us
I20260709 13:00:34.083482 10937 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.080964 11050 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:34.162597 11126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.042s
I20260709 13:00:34.180624 11021 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:33.382627 11057 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.791s
I20260709 13:00:34.161978 11090 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.192296  8749 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:34.192140  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.232192  8890 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:34.162374 11005 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:34.194043  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.233088  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1628) took 1703 ms. Trace:
I20260709 13:00:34.194818 11012 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:34.201874  8892 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:34.201895  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.263700  8891 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:34.209827  8821 rpcz_store.cc:276] 0709 13:00:32.332220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.098244 (+1766024us) service_pool.cc:224] Handling call
0709 13:00:34.098641 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.209030 (+110389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.209238 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.264966  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.083961 11094 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.213671  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.205940 11189 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:34.223213  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 314206us
I20260709 13:00:34.224884  8743 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:34.271727  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1313) took 1512 ms. Trace:
I20260709 13:00:34.272133  8743 rpcz_store.cc:276] 0709 13:00:32.759261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.224142 (+1464881us) service_pool.cc:224] Handling call
0709 13:00:34.224510 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.271504 (+ 46994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.271678 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.226018  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.272948  8747 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:34.273317  8822 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:34.273746  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1312) took 1514 ms. Trace:
I20260709 13:00:34.274442  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.196735 11173 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.233340  8889 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:34.197113  8817 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:34.234843  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 366) took 2027 ms. Trace:
I20260709 13:00:34.235378 11005 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.241389 11117 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.195644 10943 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:34.250531 11090 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.189755 11178 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:34.257658  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1615) took 1781 ms (client timeout 2209 ms). Trace:
I20260709 13:00:34.258279  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:34.259476  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.332127  8787 connection.cc:256] RPC call timeout handler was delayed by -1.23000654599997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 13:00:34.200374 11183 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:34.267354  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53823us
I20260709 13:00:34.432180  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.292435  8815 rpcz_store.cc:276] 0709 13:00:32.207982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.211901 (+  3919us) service_pool.cc:224] Handling call
0709 13:00:33.830539 (+1618638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.161467 (+330928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.234630 (+ 73163us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54003e30d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.234780 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3591851392}
I20260709 13:00:33.954026 11004 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:34.267835  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.274214  8747 rpcz_store.cc:276] 0709 13:00:32.758956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.223461 (+1464505us) service_pool.cc:224] Handling call
0709 13:00:34.269885 (+ 46424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.273459 (+  3574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.273685 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.191829 11043 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:34.427950 10814 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:34.211023 11160 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:34.231990 11126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.043s
I20260709 13:00:34.192036 11172 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.266621 11021 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.278255  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.195204 10947 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.288326  8743 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:34.233757 11166 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.233543  8744 rpcz_store.cc:276] 0709 13:00:32.529586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.206058 (+1676472us) service_pool.cc:224] Handling call
0709 13:00:34.232658 (+ 26600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.232805 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.232827 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.232862 (+    35us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:34.232893 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.232944 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.233036 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.222903 10937 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:34.290942  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.294504  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.439577 10814 tablet_bootstrap.cc:492] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 13:00:34.275722 11057 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:34.293751 11088 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:34.328504  8746 rpcz_store.cc:269] 0709 13:00:32.476254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.013791 (+537537us) service_pool.cc:224] Handling call
0709 13:00:34.257385 (+1243594us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001c7d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.257577 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701500160}
I20260709 13:00:34.440785 11117 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.440819 11005 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.213557 11054 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.276022 11094 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.260604 11050 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.279037 11189 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.398113 11104 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:34.399057  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:34.399575  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.402964  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.436015 11004 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.229348 11118 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.290586 11173 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.324582 11177 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:00:34.406548  8896 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:00:34.406829  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1627) took 1878 ms. Trace:
I20260709 13:00:34.585628  8896 pending_rounds.cc:85] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 6
I20260709 13:00:34.586072  8896 pending_rounds.cc:107] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:00:34.585732  8749 rpcz_store.cc:276] 0709 13:00:32.528989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.158569 (+1629580us) service_pool.cc:224] Handling call
0709 13:00:34.158844 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.231766 (+ 72922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.406531 (+174765us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b540013bad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:34.406745 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76539136}
I20260709 13:00:34.430379  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1632) took 1699 ms. Trace:
I20260709 13:00:34.205583 11194 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.396919 11193 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:34.399791 11012 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.326609 10943 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:34.449440  8820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.030s	sys 0.058s
W20260709 13:00:34.601581  8820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.031s	sys 0.058s
W20260709 13:00:34.435796  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 295006us
I20260709 13:00:34.460484  8822 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:34.466257  8747 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:34.597770 11055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.001s	sys 0.014s
W20260709 13:00:34.449083 11225 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:00:34.265689  8821 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:34.450667 11121 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.229099 11108 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.451297 10903 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.228998 11146 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.437834  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.443581 11178 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.459769 10994 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.633376  8815 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.491966  9414 leader_election.cc:400] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8.
W20260709 13:00:34.545979  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 362964us
I20260709 13:00:34.634390  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 367) took 2426 ms. Trace:
I20260709 13:00:34.484357 10937 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.403362  8816 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:34.634840  8823 rpcz_store.cc:276] 0709 13:00:32.208335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.208680 (+   345us) service_pool.cc:224] Handling call
0709 13:00:33.690587 (+1481907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.064160 (+373573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.633124 (+568964us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400075b50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.634312 (+  1188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3319756032}
I20260709 13:00:34.489019 11088 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.567672 11043 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.455096 11160 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.467705 11021 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.586485  8896 raft_consensus.cc:2889] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:34.376761  8893 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:34.648128  8898 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:34.586534  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:34.648785  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:34.649101  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62676.9us
I20260709 13:00:34.649362  8889 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.649801  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:34.587056  8740 rpcz_store.cc:276] 0709 13:00:32.731483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.075649 (+344166us) service_pool.cc:224] Handling call
0709 13:00:34.430118 (+1354469us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540013c4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.430305 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312044928}
W20260709 13:00:34.587200  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 391134us
I20260709 13:00:34.598758 11194 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:34.584434  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 372898us
I20260709 13:00:34.599411  8749 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:34.585552 10814 ts_tablet_manager.cc:1403] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 3.899s	user 0.037s	sys 0.028s
I20260709 13:00:34.461532  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1318) took 1251 ms. Trace:
I20260709 13:00:34.664116  8743 rpcz_store.cc:276] 0709 13:00:33.210204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.272972 (+1062768us) service_pool.cc:224] Handling call
0709 13:00:34.288122 (+ 15150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.461256 (+173134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.461480 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.613233  8746 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:34.491884 11057 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:34.667086  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.613049  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:34.613593  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.019s	sys 0.003s
W20260709 13:00:34.614477  9412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.043s	sys 0.058s
W20260709 13:00:34.668054  9412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.043s	sys 0.058s
I20260709 13:00:34.614051  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 368) took 2362 ms. Trace:
W20260709 13:00:34.597937 11153 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:00:34.616555  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.017s	sys 0.066s
I20260709 13:00:34.612286 11112 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:34.616513  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1319) took 1314 ms. Trace:
I20260709 13:00:34.619632  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:34.624022  8822 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:34.671558  8747 rpcz_store.cc:276] 0709 13:00:33.302501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.453912 (+1151411us) service_pool.cc:224] Handling call
0709 13:00:34.465872 (+ 11960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.616289 (+150417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.616465 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:34.626427  8748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.024s	sys 0.071s
I20260709 13:00:34.613330 11177 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.620306 10943 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:34.630122  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 567) took 3225 ms (client timeout 2140 ms). Trace:
I20260709 13:00:34.636816  8890 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:34.653836 10900 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.588332 11104 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:34.639860  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 148125us
I20260709 13:00:34.581492  8819 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:34.707319  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.622707 11012 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.633647 10953 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:34.465767 11183 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.648785  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.631999 11121 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.654693  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.655412  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.656154  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.663234  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1599) took 3078 ms. Trace:
I20260709 13:00:34.665984  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.657439  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1311) took 2048 ms. Trace:
W20260709 13:00:34.667994  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.019s	sys 0.004s
I20260709 13:00:34.483034  8817 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:34.717983  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 635350us
I20260709 13:00:34.693961 11235 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.668543  8820 rpcz_store.cc:276] 0709 13:00:32.251960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.362853 (+1110893us) service_pool.cc:224] Handling call
0709 13:00:33.363128 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.363325 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:33.561440 (+198115us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.601983 (+1040543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.613780 (+ 11797us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:34.613825 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.613886 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.614001 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":828359,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":87,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2430,"thread_start_us":1212872,"threads_started":2,"wal-append.queue_time_us":1020508}
W20260709 13:00:34.671146  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.017s	sys 0.066s
I20260709 13:00:34.731382  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:34.731801  8862 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.140s (SENT)
I20260709 13:00:34.732403  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:34.733096  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:34.672055  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 374) took 2046 ms. Trace:
I20260709 13:00:34.734457  8822 rpcz_store.cc:276] 0709 13:00:32.625456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.623040 (+1997584us) service_pool.cc:224] Handling call
0709 13:00:34.623380 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.671788 (+ 48408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.671998 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.574760 11228 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:00:34.685577 11237 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:34.664196 11011 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.685976 11172 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.686177 11094 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.646687 11088 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.687733  8747 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 2
I20260709 13:00:34.679411 11112 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:34.663666 11117 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:00:34.756810 11112 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.757653 11117 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:34.757886 11112 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.758085 11117 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:34.689575  9412 connection.cc:256] RPC call timeout handler was delayed by -1.9008765689999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:00:34.759253  9412 connection.cc:256] RPC call timeout handler was delayed by -1.6632765690000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:00:34.690008  8889 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:34.759778  9412 connection.cc:256] RPC call timeout handler was delayed by -1.4958765690000178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 13:00:34.760067  9412 connection.cc:256] RPC call timeout handler was delayed by -1.3932765690000224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
W20260709 13:00:34.760335  9412 connection.cc:256] RPC call timeout handler was delayed by -1.390576569000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:00:34.760381  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:34.690593  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77636us
I20260709 13:00:34.670575 11240 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:00:34.673511 11230 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:00:34.692605  8748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.026s	sys 0.072s
I20260709 13:00:34.688436  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:34.633666 10994 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:34.706002  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 86526.9us
I20260709 13:00:34.708585 11043 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.710175  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.713065 10900 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.714485  8893 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:34.653682  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.777992  8890 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:34.712118  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:34.778652  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.778837  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66809.3us
I20260709 13:00:34.713908  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.717221  8895 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.717195  8896 rpcz_store.cc:276] 0709 13:00:31.570149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.570283 (+   134us) service_pool.cc:224] Handling call
0709 13:00:31.582146 (+ 11863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.383758 (+1801612us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400379720. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6444a1d 00007f90f642b1a4 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419
0709 13:00:34.647995 (+1264237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.648164 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694158080}
I20260709 13:00:34.781518  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:34.781798  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48820.6us
I20260709 13:00:34.782296  8896 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:34.668927 11107 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:34.782655  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.727918  8817 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:34.735759  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 81373.2us
I20260709 13:00:34.743944 11011 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.788600  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:34.789366  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.791594  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.783985  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.742862  8822 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:34.793020  8890 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:34.747249  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.793370  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:34.752142  8898 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:34.753165  8747 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:34.794030  8892 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:34.794378  8747 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:34.761663  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.663899 11178 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.736340 11074 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.774866  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:34.739481 10998 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:34.705519 11233 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.751030 11005 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:34.787180  8819 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:34.793632  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1936) took 1462 ms. Trace:
I20260709 13:00:34.748232  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:34.714023 11238 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.794710  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.711666 11177 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.795876  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.759174 10992 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:34.785599 11094 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:34.709360  9444 rpcz_store.cc:269] 0709 13:00:31.395270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.398727 (+  3457us) service_pool.cc:224] Handling call
0709 13:00:34.614463 (+3215736us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54006a8954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.620176 (+  5713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506128768}
W20260709 13:00:34.804059  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.805919  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.807850  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.832166  8896 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:34.798730  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:34.813476  8822 rpcz_store.cc:276] 0709 13:00:33.331009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.742073 (+1411064us) service_pool.cc:224] Handling call
0709 13:00:34.742520 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.793396 (+ 50876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.793583 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.780778 11245 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.741714 11235 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.762768 11180 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:34.781963 10947 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.788349 11055 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:34.797996 11011 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.818128  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:34.848539  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.855472  9448 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:34.798903 11178 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.808446  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:34.842675  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.843406  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:34.839869  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:34.815487  8817 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:34.816368 11005 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.811133 11172 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.811879 11021 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.816926  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:34.814397 11054 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.821472 11173 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.855984  8819 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:34.818593 11108 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.870663 11238 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:34.827452 11160 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.827065 11057 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.754669 11237 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.010223 10937 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.010335 11238 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.011886 11237 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.840139 10987 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.012552 10937 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.012629 10987 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:34.857831  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 169542us
I20260709 13:00:35.014357 11177 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.014575  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:35.015429 10994 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.859479  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:34.783635 11117 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.017831 11117 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.025559  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.033239  8817 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:35.034282  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.917075  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:34.861536 11249 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:34.854636 11055 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.056406  8817 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:35.057016  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1938) took 1725 ms. Trace:
I20260709 13:00:35.057318  8817 rpcz_store.cc:276] 0709 13:00:33.331452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.055900 (+1724448us) service_pool.cc:224] Handling call
0709 13:00:35.056201 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.056799 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.056968 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:34.849058 11246 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.872056  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 370) took 2569 ms. Trace:
I20260709 13:00:35.126511  8816 rpcz_store.cc:276] 0709 13:00:32.302454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.086432 (+1783978us) service_pool.cc:224] Handling call
0709 13:00:34.086667 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.639074 (+552407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.871798 (+232724us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:34.871987 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728164096}
I20260709 13:00:35.127641  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.914620  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1939) took 1424 ms. Trace:
I20260709 13:00:34.897272  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1316) took 1759 ms. Trace:
I20260709 13:00:35.129262  8745 rpcz_store.cc:276] 0709 13:00:33.137811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.230460 (+1092649us) service_pool.cc:224] Handling call
0709 13:00:34.230804 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.595607 (+364803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.595651 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.595713 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:34.595770 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.896998 (+301228us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54000b4d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:34.897057 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.897201 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838795520}
I20260709 13:00:34.717576  8742 rpcz_store.cc:276] 0709 13:00:32.605384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.206648 (+1601264us) service_pool.cc:224] Handling call
0709 13:00:34.206985 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.207222 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:34.596225 (+389003us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.653119 (+ 56894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.653194 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:34.653256 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.653363 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.653496 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":420660,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":4295,"raft.queue_time_us":63525,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5180,"thread_start_us":407503,"threads_started":2}
I20260709 13:00:34.832612 11094 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:34.883143  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:34.998500  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 378) took 2293 ms. Trace:
I20260709 13:00:35.137699  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:35.137905  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 278641us
I20260709 13:00:35.138768  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.848306 11245 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:35.000469 11244 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:00:34.902989 10900 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.880452 11050 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.896476 11090 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.923373 11121 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:34.884359  9440 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:35.135030 11021 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.022020 11183 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:35.003182 11243 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:00:34.815959 11233 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:34.934043  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 379) took 2053 ms. Trace:
I20260709 13:00:35.203946  8823 rpcz_store.cc:276] 0709 13:00:32.880464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.639346 (+1758882us) service_pool.cc:224] Handling call
0709 13:00:34.647350 (+  8004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.737410 (+ 90060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.737438 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.737480 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:34.737531 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.933765 (+196234us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:34.933822 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.933973 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206757248}
I20260709 13:00:34.925858  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:35.094254  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.133246 10947 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:00:35.123407  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1624) took 2644 ms. Trace:
I20260709 13:00:35.125393  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:35.132752  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.134241  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:34.859767 11178 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.134495  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:34.926733 10968 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.919778  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 380) took 1893 ms. Trace:
I20260709 13:00:35.128624  8818 rpcz_store.cc:276] 0709 13:00:33.490621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.696661 (+206040us) service_pool.cc:224] Handling call
0709 13:00:34.914373 (+1217712us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54001b8854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.914542 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786595072}
I20260709 13:00:34.998771 11240 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.140504  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:34.900880 11004 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.963631 11005 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:35.146526  8819 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:00:35.148157  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 289396us
I20260709 13:00:35.023716  9444 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 2
I20260709 13:00:35.023162 11180 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:34.857393 11189 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.018971 10987 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.055228 10992 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:35.018308 11194 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.054847 11154 raft_consensus.cc:2764] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:00:35.166236  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 404823us
W20260709 13:00:35.164641  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 390039us
I20260709 13:00:35.124411 11055 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.124640 11170 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:35.166774  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 329879us
W20260709 13:00:35.169060  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 376031us
I20260709 13:00:35.131762 11246 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:34.890153  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:34.867549 11118 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:35.132121 11172 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:35.059204  9441 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:35.136548 11177 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.198006  8742 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:35.164248 10953 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:35.166519 11104 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:34.854396 11228 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.148681 11173 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:00:35.209466  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1322) took 1051 ms. Trace:
I20260709 13:00:35.136052 11088 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.197041 11074 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
W20260709 13:00:35.328084 11112 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.001s
I20260709 13:00:35.202884 11021 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:35.318882 11269 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:00:35.347697 11233 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.371579 10953 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.372671 10992 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:35.373724 10992 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.374200  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.374872 11088 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.375488  8744 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:35.376068  8820 rpcz_store.cc:276] 0709 13:00:33.026779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.730603 (+1703824us) service_pool.cc:224] Handling call
0709 13:00:34.917250 (+186647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.917517 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.917564 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.917621 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:34.917669 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.917740 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.919564 (+  1824us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54001b8850. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.919706 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380958464}
I20260709 13:00:35.377163 11246 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.377400  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 331) took 1320 ms. Trace:
I20260709 13:00:35.377686  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.377805  8744 rpcz_store.cc:276] 0709 13:00:34.056589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.568451 (+511862us) service_pool.cc:224] Handling call
0709 13:00:34.579366 (+ 10915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.376433 (+797067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.377172 (+   739us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b540013c4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:35.377346 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1831385984}
I20260709 13:00:35.379241  8744 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:35.208738  8748 rpcz_store.cc:276] 0709 13:00:32.478925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.514022 (+1035097us) service_pool.cc:224] Handling call
0709 13:00:34.100479 (+586457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.769931 (+669452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.769964 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.777898 (+  7934us) log.cc:844] Serialized 26 byte log entry
0709 13:00:35.037533 (+259635us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:35.037591 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.037680 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:35.123045 (+ 85365us) raft_consensus.cc:1685] finished
0709 13:00:35.123080 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.123302 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53022,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":891750,"threads_started":2,"wal-append.queue_time_us":322195}
W20260709 13:00:35.324002 10943 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.000s
I20260709 13:00:35.381412  8745 rpcz_store.cc:276] 0709 13:00:34.156791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.130470 (+973679us) service_pool.cc:224] Handling call
0709 13:00:35.130745 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.207924 (+ 77179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.207958 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.208064 (+   106us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:35.208118 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.208203 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.208376 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19287,"raft.queue_time_us":46898,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4245,"thread_start_us":57093,"threads_started":1}
I20260709 13:00:35.383075 11228 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.383533 11228 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.177807 11253 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.383942 11253 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.384269 11253 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.386900 11074 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:35.297915 11183 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.387394 11074 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:35.388293 11074 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.389550  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:35.390710 11233 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:35.371574  8891 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:35.391353 11233 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:35.392252 11233 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.136917  8821 rpcz_store.cc:276] 0709 13:00:32.705040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.628737 (+1923697us) service_pool.cc:224] Handling call
0709 13:00:34.711268 (+ 82531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.711523 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:34.739850 (+ 28327us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.963135 (+223285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.963192 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:34.963231 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.963288 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.998329 (+ 35041us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b540029a4d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:34.998441 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101661,"prepare.queue_time_us":60501,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":190777,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":10632,"spinlock_wait_cycles":195808384,"thread_start_us":123693,"threads_started":1,"wal-append.queue_time_us":383477}
I20260709 13:00:35.396170  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:35.396975  9415 leader_election.cc:336] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.977s (SENT)
I20260709 13:00:35.230365  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:35.215508  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81789.1us
I20260709 13:00:35.397902  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:35.398291  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:35.398665  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.397182  8821 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:35.398468  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:35.399645  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:35.401258  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:35.375779  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.404188  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:35.406064  8817 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:35.407912  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.409448  8817 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:35.414793  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:35.416096  8862 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.006s	sys 0.028s
W20260709 13:00:35.416419  8862 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.006s	sys 0.028s
I20260709 13:00:35.199437 10909 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.420255  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.373011 11021 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.423808  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:35.344013 11043 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.555s	user 0.005s	sys 0.002s
W20260709 13:00:35.344442  8815 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.001s
W20260709 13:00:35.312671 11166 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.830s	user 0.007s	sys 0.003s
W20260709 13:00:35.345212 10903 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.001s
W20260709 13:00:35.354681 11193 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.611s	user 0.004s	sys 0.004s
I20260709 13:00:35.403915  9444 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:35.311748 11235 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.323551 11271 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.213115 11245 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.455039  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.454114  8897 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:35.208976 11050 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.208066 11248 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:35.325729 11094 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.449668 10943 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.355623 11011 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.212229 10814 raft_consensus.cc:359] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:35.452847  8815 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:35.454046 11154 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:35.421768  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:35.431620  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.420758 11267 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:35.363215 11146 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.400799 11170 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.454283  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.420499 11054 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.457764 11172 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.490701 11043 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.494653 11259 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:35.495615 11259 consensus_queue.cc:1243] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:00:35.490749 11054 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.496537 11054 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.467760 11271 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.502686 11011 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.502086 11055 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.503444 11194 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.503994  9443 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:35.313464  9442 raft_consensus.cc:2376] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 3. Re-sending same reply.
I20260709 13:00:35.456236  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:35.423529 11189 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:35.505013  8818 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:35.504964 11189 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:35.423693 11180 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:35.505965  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:35.506556  8818 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.506613 11180 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.421564 11118 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.522182  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.522925  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.523454  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:35.523959 11236 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:35.524044  8893 raft_consensus.cc:2376] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6. Re-sending same reply.
I20260709 13:00:35.524298  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.524415 11236 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:35.524992  8891 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
I20260709 13:00:35.525557  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.524066  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.524992  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.454902  9444 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:35.451114 11112 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:35.527534  8817 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:35.528694  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.404520 11240 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.472430  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.537134  8744 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:35.423125 11173 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:00:35.475668  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1948) took 1372 ms. Trace:
I20260709 13:00:35.549474  8815 rpcz_store.cc:276] 0709 13:00:34.103649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.437526 (+333877us) service_pool.cc:224] Handling call
0709 13:00:35.475606 (+1038080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.550892  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.463901 11166 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.473668 10943 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:35.450583 11057 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:00:35.450233 10937 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:00:35.480530  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:35.430158  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:35.483644  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61987.1us
I20260709 13:00:35.556111  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:35.557533  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77104.3us
I20260709 13:00:35.557648  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:35.558314  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:35.559141  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:35.559590  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 129634us
I20260709 13:00:35.421150 11177 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.484802 11170 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.473891 10814 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 13:00:35.513181  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.514422  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:35.516712  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.505976 11189 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.454684  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:35.527550  8896 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:35.529954  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.424466 11108 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:35.566004  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 146343us
W20260709 13:00:35.470335  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1621715590000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22740000000000002s for a response
I20260709 13:00:35.571623  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:35.534417 11112 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.567262  8890 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:35.574010  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:35.534958  8818 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:35.536191  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:35.470381 11050 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.477650 11264 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.552310  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.576845  8817 raft_consensus.cc:2376] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6. Re-sending same reply.
I20260709 13:00:35.559725  8889 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:35.560376  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.562645  8740 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:35.577201  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:35.581547  8815 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:35.581753  8893 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:35.557224  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 142920us
I20260709 13:00:35.564126  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.586108  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:35.586793  9415 leader_election.cc:336] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 2.094s (SENT)
I20260709 13:00:35.564515  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:35.588181  8748 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.565205  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:35.483310 11154 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.553365 11240 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.566939  8895 raft_consensus.cc:2376] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6. Re-sending same reply.
I20260709 13:00:35.510495  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:35.549393 11146 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.582178  8817 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.582446  8744 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:35.504077 11194 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.592036  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.598982  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1949) took 1495 ms. Trace:
I20260709 13:00:35.601527  8817 rpcz_store.cc:276] 0709 13:00:34.103909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.581721 (+1477812us) service_pool.cc:224] Handling call
0709 13:00:35.582012 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.598751 (+ 16739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.598930 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.603036  8817 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:35.592345  8890 raft_consensus.cc:2376] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5. Re-sending same reply.
W20260709 13:00:35.596410  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 176851us
I20260709 13:00:35.572587 11140 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.567500 10900 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:35.604247  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.610281  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:35.611120  8744 raft_consensus.cc:2376] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7. Re-sending same reply.
I20260709 13:00:35.568377 11166 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:35.567742 11173 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:35.567392 11021 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:35.606310  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.549154 11253 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.579756 11112 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.610196  8746 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:35.616066  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1304) took 1454 ms. Trace:
I20260709 13:00:35.569509 11255 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:35.581104 11260 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.482502 11248 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.593199 11004 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.621344  8817 rpcz_store.cc:276] 0709 13:00:34.151903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.602471 (+1450568us) service_pool.cc:224] Handling call
0709 13:00:35.602707 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.605492 (+  2785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.606413 (+   921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.626224  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.598905 11183 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:35.625120  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65619.6us
I20260709 13:00:35.625711  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.628049  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:35.631553  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60030.4us
I20260709 13:00:35.632678 11288 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.633320 11288 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.632654  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:35.628099  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 117746us
I20260709 13:00:35.566612 11270 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:00:35.583214 10937 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:00:35.568687 11011 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:35.580876 11043 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:00:35.621654 11166 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.616900 10900 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.640640  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.612447 11278 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:35.650132  8748 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:35.604440 11146 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.683643  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.633658 11288 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.669634  8817 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:35.638659 11173 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:35.568378 11251 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:00:35.692404  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.677973  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.693619  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.693763  8744 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:35.679268  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.686264  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1307) took 1533 ms. Trace:
I20260709 13:00:35.580722 11250 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:00:35.614117 11140 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.694180  8748 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
W20260709 13:00:35.691435  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81463.4us
I20260709 13:00:35.694978  8817 rpcz_store.cc:276] 0709 13:00:34.152676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.627909 (+1475233us) service_pool.cc:224] Handling call
0709 13:00:35.669347 (+ 41438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.685978 (+ 16631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.686192 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.642585 11277 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.694648  8896 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:35.650395 11021 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:35.593046 11264 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:35.699201  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 93277.1us
I20260709 13:00:35.705993 11250 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.679272 11117 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.700630  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.707528  8748 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:35.708081  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.708341  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.683534 10900 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.680074 11166 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.706430 11250 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:35.710130  8745 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:35.710995  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.713958  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.714236  8744 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:35.667812 11273 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.714643  8748 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:35.682893 11278 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:35.722523 11278 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.723862 11278 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:35.609519 10814 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:35.733668 11004 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:35.734462 11004 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.737600  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.706943 11117 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.802425  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.704499 11277 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:35.638948 11246 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:35.641597 11260 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.742946 11055 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.828333 11117 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:35.833618 11250 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.834432 11074 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.835681 11267 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.836061 11267 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:35.841789 10987 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.842983  8822 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:35.843201  8816 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:35.843696  8816 pending_rounds.cc:85] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:35.843937  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1954) took 1067 ms. Trace:
I20260709 13:00:35.844323  8822 rpcz_store.cc:276] 0709 13:00:34.776104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.839627 (+ 63523us) service_pool.cc:224] Handling call
0709 13:00:35.843741 (+1004114us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:35.843883 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265091968}
I20260709 13:00:35.700529 11281 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.639081 11282 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:00:35.785166 11021 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.789064  8889 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:35.848423  8822 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:35.849221  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1312) took 1661 ms. Trace:
I20260709 13:00:35.843973  8816 pending_rounds.cc:107] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:35.850131  8816 raft_consensus.cc:2889] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:35.849575  8822 rpcz_store.cc:276] 0709 13:00:34.187471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.845113 (+1657642us) service_pool.cc:224] Handling call
0709 13:00:35.848264 (+  3151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.848924 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.849166 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.851634  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:35.691865 11285 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:00:35.785719 11057 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:35.750232  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:35.815747 11283 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:00:35.639842 11255 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:35.692042 11108 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.058s
W20260709 13:00:35.815991  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.844813  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:35.794651  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 162169us
I20260709 13:00:35.856457  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:35.871294  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:35.859477  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1950) took 1417 ms. Trace:
I20260709 13:00:35.859654  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:35.860807  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:35.873447  8895 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.876175  8822 rpcz_store.cc:276] 0709 13:00:34.434622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.850879 (+1416257us) service_pool.cc:224] Handling call
0709 13:00:35.851163 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.851326 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.851350 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.851386 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:35.851419 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.851468 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.851551 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.800851  8748 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:35.861724  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1311) took 1677 ms. Trace:
I20260709 13:00:35.877641  8816 rpcz_store.cc:276] 0709 13:00:34.183991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.842382 (+1658391us) service_pool.cc:224] Handling call
0709 13:00:35.842757 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.861438 (+ 18681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.861658 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.887920  8895 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:35.861248  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111132us
I20260709 13:00:35.889423  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:35.893630  8822 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.814047 11277 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.867648 11011 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:35.836838 11267 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.896759  8818 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:36.123915  8816 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:36.124816  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1315) took 1439 ms. Trace:
I20260709 13:00:36.125236  8816 rpcz_store.cc:276] 0709 13:00:34.685766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.895310 (+1209544us) service_pool.cc:224] Handling call
0709 13:00:36.099967 (+204657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.124504 (+ 24537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.124745 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.867204 11235 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.841045 10900 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:35.868759 11279 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:00:35.825380 11005 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.895870 11104 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:36.170039  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 298961us
I20260709 13:00:36.171311  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:35.927717  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1314) took 1251 ms. Trace:
I20260709 13:00:36.172216  8822 rpcz_store.cc:276] 0709 13:00:34.675999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.892584 (+1216585us) service_pool.cc:224] Handling call
0709 13:00:35.893269 (+   685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.927276 (+ 34007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.927622 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.926952  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:36.173918  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 279762us
I20260709 13:00:35.751761 11273 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:35.932085  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 342) took 1101 ms. Trace:
I20260709 13:00:36.190836  8743 rpcz_store.cc:276] 0709 13:00:34.830521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.298286 (+467765us) service_pool.cc:224] Handling call
0709 13:00:35.298627 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.669449 (+370822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.669480 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.669521 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:35.669554 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.931781 (+262227us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400160dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:35.931847 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.932012 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852715904}
I20260709 13:00:35.878526 11177 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:35.895395 11088 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:36.101104 11292 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
W20260709 13:00:36.168111  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 466465us
I20260709 13:00:35.834851 11250 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.119890  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:35.891626 11043 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.272s
I20260709 13:00:36.195107  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:36.161407 11154 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.161799 11194 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:35.895040 11251 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.269s
I20260709 13:00:35.895397 11259 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.269s
I20260709 13:00:35.926259 11291 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.307s
I20260709 13:00:35.863163 11173 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:35.891391 10943 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:36.195948  8893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.064s
I20260709 13:00:36.249555  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:36.249866  8893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.000s	sys 0.064s
I20260709 13:00:36.250684  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.250885  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.251818  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.252038  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.254096 10900 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.234313 11104 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.255108 11104 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.226410 11088 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:36.120803 11284 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.303s
I20260709 13:00:36.199154 11282 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.163430 11235 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:36.228078  8820 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:36.257266 11104 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:36.259774 10968 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.845s	user 0.007s	sys 0.003s
I20260709 13:00:36.260035  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.275279  9445 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.277164  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 585) took 1509 ms. Trace:
I20260709 13:00:36.278461 11073 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.261485 11088 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.279116 11073 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.229816  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1634) took 2072 ms. Trace:
I20260709 13:00:36.280747  8742 rpcz_store.cc:276] 0709 13:00:34.157054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.131683 (+974629us) service_pool.cc:224] Handling call
0709 13:00:35.131992 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.396836 (+264844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.229552 (+832716us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:36.229741 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151120640}
I20260709 13:00:36.282809  8741 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:36.284426  8742 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:36.286540  8822 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:36.286682  8742 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:36.263616  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1321) took 2107 ms. Trace:
I20260709 13:00:36.265353  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.288329 11177 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.289062 11177 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.193019 11233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.009s	sys 0.009s
I20260709 13:00:36.233135  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 1937) took 2901 ms. Trace:
W20260709 13:00:36.209040  8788 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.522s (SENT)
I20260709 13:00:36.194944 11267 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.195314 11245 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:36.238415  8816 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:36.241098  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:36.242820  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1296) took 4389 ms (client timeout 2176 ms). Trace:
W20260709 13:00:36.302363  8824 rpcz_store.cc:269] 0709 13:00:31.842196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.914233 (+ 72037us) service_pool.cc:224] Handling call
0709 13:00:36.231337 (+4317104us) spinlock_profiling.cc:243] Waited 3.48 s on lock 0x7b5400076554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:36.231440 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8002538240}
I20260709 13:00:36.231294 11050 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.244202 10987 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.245770  9440 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:36.246845  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 341) took 2107 ms. Trace:
W20260709 13:00:36.248245 11178 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.827s	user 0.006s	sys 0.002s
W20260709 13:00:36.171195 11280 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 13:00:36.297875  8819 rpcz_store.cc:276] 0709 13:00:33.331229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.008037 (+1676808us) service_pool.cc:224] Handling call
0709 13:00:35.008353 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.400936 (+392583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.232907 (+831971us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540004ddd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:36.233083 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317388032}
I20260709 13:00:36.264768 11246 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.266592 11005 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:36.268365 11054 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.771s	user 0.003s	sys 0.008s
I20260709 13:00:36.308554  8740 rpcz_store.cc:276] 0709 13:00:34.139446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.672170 (+532724us) service_pool.cc:224] Handling call
0709 13:00:34.709984 (+ 37814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.582615 (+872631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.246616 (+664001us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540013b5d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:36.246787 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961360768}
I20260709 13:00:36.334626  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1631) took 1692 ms. Trace:
W20260709 13:00:36.297713 11233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.010s	sys 0.010s
I20260709 13:00:36.272761 11235 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.272358 11293 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:36.272190 11273 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.270395 11264 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:36.268182 10814 ts_tablet_manager.cc:1434] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 1.593s	user 0.011s	sys 0.012s
I20260709 13:00:36.272096 11284 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:36.269769 11251 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:36.267578 11296 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:36.262357 11276 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.414s	user 0.000s	sys 0.000s
I20260709 13:00:36.298110  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.299190 10900 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:36.347960 11278 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.000s
W20260709 13:00:36.335253  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 584) took 1568 ms (client timeout 1925 ms). Trace:
W20260709 13:00:36.346352 11228 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.962s	user 0.012s	sys 0.000s
I20260709 13:00:36.396610  8897 rpcz_store.cc:276] 0709 13:00:34.642373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.651027 (+  8654us) service_pool.cc:224] Handling call
0709 13:00:36.334444 (+1683417us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54000947d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:36.334559 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694829312}
I20260709 13:00:36.393683 11178 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.270412 11108 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.288367  8747 rpcz_store.cc:276] 0709 13:00:34.156545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.672706 (+516161us) service_pool.cc:224] Handling call
0709 13:00:34.687542 (+ 14836us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:35.459617 (+772075us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b18000618a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64c7bfb 00007f90f64d264b
0709 13:00:35.459778 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.459807 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:35.460430 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 13:00:35.569361 (+108931us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:35.569396 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.569474 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:36.263258 (+693784us) raft_consensus.cc:1685] finished
0709 13:00:36.263283 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.263551 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40088,"raft.queue_time_us":58293,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":107869,"spinlock_wait_cycles":37756160,"thread_start_us":342066,"threads_started":2,"wal-append.queue_time_us":220249}
W20260709 13:00:36.358681 11189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.020s	sys 0.024s
I20260709 13:00:36.420102 11284 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:36.401821  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.271274 11260 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.273514  9441 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:36.329538 11177 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.331995 11004 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.340333 11146 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.341059 11245 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:36.384002 11050 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.341174 11271 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.460719 11296 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:36.423689 11277 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.384245  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 281924us
I20260709 13:00:36.404525  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:36.427767  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.039s	sys 0.034s
W20260709 13:00:36.428063  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 219184us
W20260709 13:00:36.465135 11021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.021s	sys 0.056s
W20260709 13:00:36.477311  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.039s	sys 0.034s
W20260709 13:00:36.478781  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 619494us
I20260709 13:00:36.479552  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:36.419227 11170 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.846s	user 0.005s	sys 0.005s
I20260709 13:00:36.401624 11293 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:36.398532  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:36.430502  9443 rpcz_store.cc:269] 0709 13:00:34.767138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.797044 (+ 29906us) service_pool.cc:224] Handling call
0709 13:00:34.889960 (+ 92916us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:36.335060 (+1445100us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54005b09d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:36.335203 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252366080}
I20260709 13:00:36.424762 11251 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:36.435586 11248 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.443132 11073 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.342054 11236 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.444653  8741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1619) took 3967 ms (client timeout 2274 ms). Trace:
W20260709 13:00:36.446326 11112 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.804s	user 0.007s	sys 0.003s
W20260709 13:00:36.449203 11240 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.839s	user 0.008s	sys 0.002s
I20260709 13:00:36.429644 11005 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.450348  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 339) took 2311 ms. Trace:
I20260709 13:00:36.453409 11228 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.455674 11074 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.004s
I20260709 13:00:36.532606  8749 rpcz_store.cc:276] 0709 13:00:34.138818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.662500 (+523682us) service_pool.cc:224] Handling call
0709 13:00:34.689962 (+ 27462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.017684 (+327722us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:35.017968 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:00:35.403960 (+385992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.404007 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:35.404051 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.450055 (+1046004us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400464bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:36.450117 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.450280 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129997,"prepare.queue_time_us":577,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":53867,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":4439,"spinlock_wait_cycles":753432448,"thread_start_us":359922,"threads_started":2,"wal-append.queue_time_us":158184,"wal-append.run_cpu_time_us":9310,"wal-append.run_wall_time_us":1205826}
I20260709 13:00:36.432915 10900 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.277810  9444 rpcz_store.cc:276] 0709 13:00:34.767447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.857336 (+ 89889us) service_pool.cc:224] Handling call
0709 13:00:35.023470 (+166134us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:35.607659 (+584189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.613445 (+  5786us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:35.704051 (+ 90606us) log.cc:844] Serialized 52 byte log entry
0709 13:00:35.919982 (+215931us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:36.101265 (+181283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.132872 (+ 31607us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:36.276830 (+143958us) raft_consensus.cc:1685] finished
0709 13:00:36.276865 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.277076 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14937,"raft.queue_time_us":69641,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":17273,"thread_start_us":113371,"threads_started":1,"wal-append.queue_time_us":565569}
I20260709 13:00:36.486505 11011 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:36.462682 11189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.021s	sys 0.025s
I20260709 13:00:36.472092  8821 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:36.411377 10814 ts_tablet_manager.cc:1467] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Tablet startup took 1747ms
I20260709 13:00:36.402751 11054 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.479575  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:36.568773  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89333.3us
W20260709 13:00:36.484483  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 309300us
I20260709 13:00:36.533195 11140 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.485261  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:36.482978  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 283264us
I20260709 13:00:36.488607  9449 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:00:36.478235 11021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.021s	sys 0.056s
W20260709 13:00:36.524740  8862 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.090s (SENT)
I20260709 13:00:36.527833  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.571944  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:36.552768 11236 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.540670  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.487084 11168 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:36.463526  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:36.575057  8819 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:36.485214 11170 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:36.575762  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112399us
I20260709 13:00:36.576182  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.576761  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:36.540171 11112 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:36.558909 11294 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.124s	user 0.005s	sys 0.000s
I20260709 13:00:36.540634 11240 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.566812  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.589500  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.477733 11050 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.571537  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:36.529541  8741 rpcz_store.cc:269] 0709 13:00:32.477452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.053456 (+576004us) service_pool.cc:224] Handling call
0709 13:00:36.444472 (+3391016us) spinlock_profiling.cc:243] Waited 3.23 s on lock 0x7b540016e9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:36.444579 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7428323200}
I20260709 13:00:36.572856  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:36.591073  8822 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:36.591391  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:36.591804  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.592001  8741 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:36.592926  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:36.593354  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.577693  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:36.572007  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.529347 11246 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:36.573948  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:36.593439  8741 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:36.586928  9414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.004s	sys 0.017s
I20260709 13:00:36.573716 11005 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.616915  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.617451  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:36.617719  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46257.3us
I20260709 13:00:36.619169  8824 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:36.595207  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:36.620121  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 95450.4us
I20260709 13:00:36.595505  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.621829  8819 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:36.623292  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:36.624219  9414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.005s	sys 0.017s
W20260709 13:00:36.629136  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 917773us
I20260709 13:00:36.631059  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:36.585842 11074 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.574841  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:36.624857  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 47250.3us
I20260709 13:00:36.640793  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:36.641938  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:36.586386 11073 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.637866  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:36.638654  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.594769 11140 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.643077  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.631846  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:36.643733 11310 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:36.624639  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:36.627990  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.653177  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:36.631944  8816 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:36.626873  8898 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:36.650177  8890 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:36.633630  8824 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:36.628455  8897 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:36.653786  8895 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.633482  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.657665  9448 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:36.626107 11005 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.658493  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:36.658975 11303 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:00:36.659050 11005 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.659761  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.660310 11054 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.660782  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.654204  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:36.662676  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:36.632699  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:36.646538  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:36.653262  8749 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:36.626358 11108 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:36.627956 11256 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.631511 11309 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:36.631855 11168 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.682976 11260 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.552927 11284 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.624148 11300 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.684856  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.644450 11193 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.676208  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:36.687103  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:36.682277  8741 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:36.574967 11245 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.693570  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.651561  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.651081 11180 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:00:36.636981 11233 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:00:36.651381 11270 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:00:36.618079 11251 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.675609 11238 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.665221 11074 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:36.673015 11166 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.921s	user 0.012s	sys 0.000s
I20260709 13:00:36.818218 11166 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:36.820868 11166 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.685545 11108 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.821794 11108 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:36.589607 11296 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.648027 11173 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.817597  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.825044  9446 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:00:36.825105  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 157655us
I20260709 13:00:36.825652  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:36.826463  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 196593us
W20260709 13:00:36.827174  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 169872us
I20260709 13:00:36.827427  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:36.827663  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:36.763506  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:36.828321  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 123192us
I20260709 13:00:36.828580  8896 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
W20260709 13:00:36.829159  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 201327us
I20260709 13:00:36.658687 11194 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:36.791191  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:36.829777  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:36.803789  8895 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:36.855975  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:36.821828 11304 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 13:00:36.828131  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:36.832962  8740 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:36.647365 11011 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:36.784060 11288 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.081s	user 0.012s	sys 0.000s
I20260709 13:00:36.857198  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.873040  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:36.875406  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.875556  8895 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:36.873476 11168 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:36.876432  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:36.876989  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:36.877444  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:36.721881 11287 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.675113 11088 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:36.833179 11248 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:36.748289 11297 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.317s	user 0.000s	sys 0.003s
I20260709 13:00:36.878150  8895 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:36.683601 11285 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:36.910678  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:36.885370  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:36.870250  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 183302us
I20260709 13:00:36.741442 11057 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.741225 11178 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:36.722452 11310 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:36.575202 11293 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.722538 11309 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.949141 11309 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:36.951115  8849 heartbeater.cc:646] Failed to heartbeat to 127.7.117.190:45111 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.117.190:45111 timed out after 15.000s (SENT)
I20260709 13:00:36.762197 11277 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.882146 11168 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:36.882390 11228 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:36.879309 11140 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:36.835163 11194 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:36.828423  8741 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:36.834131 11173 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:36.747705  9445 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:36.866271 11256 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.862387 11246 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:36.855649 11270 consensus_queue.cc:1243] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:36.911592  8897 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 3.
I20260709 13:00:37.066093  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:37.066820  8745 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:36.880443 11288 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.879940 11284 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:36.879441 11271 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.943953  8895 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.882011 11251 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.883505  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.081642  9440 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:37.082721  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.083998  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:36.857429 11295 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:00:36.991839 11021 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.884107 11043 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 13:00:36.859017 11193 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:36.943408 11321 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.960492 11170 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:36.967960 11316 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:00:36.909518 11074 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.925081  9444 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:36.909209 11285 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:37.042686  8862 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:37.050314  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.043452  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:36.882987 11112 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:36.913028 11088 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:37.055075  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 183071us
W20260709 13:00:37.057135 11314 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.275s	user 0.002s	sys 0.002s
W20260709 13:00:37.057494 11307 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.222s	user 0.000s	sys 0.004s
I20260709 13:00:37.052390 11173 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.058348 11168 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.047254 11194 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.061087  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 1935) took 1549 ms. Trace:
I20260709 13:00:37.073115  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:36.881194 11300 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.174s
W20260709 13:00:37.056476 11235 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.001s
I20260709 13:00:37.078366  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:37.079237  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.065055 11267 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.001s
I20260709 13:00:37.043372 11140 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.056209 11246 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:37.087450  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.089605  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 266210us
I20260709 13:00:37.091046  9440 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:37.086310 11271 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:37.069550 11005 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.107378  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.589723  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.141884  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 313949us
I20260709 13:00:37.590304  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1341) took 1007 ms. Trace:
I20260709 13:00:37.249091  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:37.596195  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 682670us
I20260709 13:00:37.589033  8741 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:37.590579  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:37.598965  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.531055  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1303) took 3373 ms. Trace:
I20260709 13:00:37.549685  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1644) took 2087 ms. Trace:
I20260709 13:00:37.530498  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1642) took 1965 ms. Trace:
I20260709 13:00:37.601274  8744 rpcz_store.cc:276] 0709 13:00:35.462748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.871444 (+408696us) service_pool.cc:224] Handling call
0709 13:00:35.920360 (+ 48916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.232335 (+311975us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:36.563923 (+331588us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.675901 (+111978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.675953 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:36.675995 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.549349 (+873354us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400073ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:37.549419 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.549601 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":399563,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":3668,"raft.queue_time_us":7589,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":744010368,"thread_start_us":411845,"threads_started":2,"wal-append.queue_time_us":194291}
W20260709 13:00:37.147639  8713 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 3.125s (SENT) [suppressed 48 similar messages]
I20260709 13:00:37.603892  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1339) took 1017 ms. Trace:
I20260709 13:00:37.604571  9447 raft_consensus.cc:2410] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:37.596522  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 526) took 1252 ms. Trace:
I20260709 13:00:37.605016  9446 rpcz_store.cc:276] 0709 13:00:36.344187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.882363 (+538176us) service_pool.cc:224] Handling call
0709 13:00:37.248896 (+366533us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:37.596435 (+347539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2779904}
I20260709 13:00:37.597388  8898 rpcz_store.cc:276] 0709 13:00:35.512084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.512953 (+   869us) service_pool.cc:224] Handling call
0709 13:00:37.060815 (+1547862us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400249354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.061008 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438416640}
W20260709 13:00:37.604334  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 456928us
I20260709 13:00:37.607388  8745 rpcz_store.cc:276] 0709 13:00:36.583202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.072407 (+489205us) service_pool.cc:224] Handling call
0709 13:00:37.072738 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.590052 (+517314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.590254 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.606017  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1655) took 1020 ms. Trace:
I20260709 13:00:37.608349  8747 rpcz_store.cc:276] 0709 13:00:36.585192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.067895 (+482703us) service_pool.cc:224] Handling call
0709 13:00:37.605925 (+538030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.526548  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1646) took 1953 ms. Trace:
I20260709 13:00:37.600612  8818 rpcz_store.cc:276] 0709 13:00:34.151656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.583767 (+1432111us) service_pool.cc:224] Handling call
0709 13:00:35.601395 (+ 17628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.122479 (+521084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.525005 (+1402526us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54002297d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:37.525225 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615854848}
I20260709 13:00:37.606056  8741 rpcz_store.cc:276] 0709 13:00:36.582714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.043187 (+460473us) service_pool.cc:224] Handling call
0709 13:00:37.600269 (+557082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.609133  8748 rpcz_store.cc:276] 0709 13:00:35.572854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.893908 (+321054us) service_pool.cc:224] Handling call
0709 13:00:36.451696 (+557788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.451882 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.451918 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.451960 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:36.451989 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.452046 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.526344 (+1074298us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400200b50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.526479 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282527104}
W20260709 13:00:37.614667  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:37.489508 11250 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.266s	user 0.013s	sys 0.002s
I20260709 13:00:37.092599 11321 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:37.596697 11172 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.006s	sys 0.006s
W20260709 13:00:37.231186 11236 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.000s
I20260709 13:00:37.622655  8741 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:37.664274  8741 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:00:37.664638  8741 pending_rounds.cc:85] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 5
I20260709 13:00:37.665082  8741 pending_rounds.cc:107] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:37.665310  8741 raft_consensus.cc:2889] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:37.666841  8741 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:37.439756 11288 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:37.669373  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.670126  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 515) took 1142 ms. Trace:
I20260709 13:00:37.670596  9446 rpcz_store.cc:276] 0709 13:00:36.527660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.622988 (+1095328us) service_pool.cc:224] Handling call
0709 13:00:37.670072 (+ 47084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.247103 11253 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:37.456187  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 575) took 2695 ms (client timeout 2140 ms). Trace:
I20260709 13:00:37.598995 11285 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:37.590885 11320 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.506s	user 0.001s	sys 0.000s
W20260709 13:00:37.511960 11238 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.002s
I20260709 13:00:37.602105  8891 rpcz_store.cc:276] 0709 13:00:35.564717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.602629 (+ 37912us) service_pool.cc:224] Handling call
0709 13:00:35.615536 (+ 12907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.615813 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:35.638120 (+ 22307us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.529863 (+1891743us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18003624a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:00:37.530098 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.530159 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:37.530219 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.530307 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.530431 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22409,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":6797,"raft.queue_time_us":884,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":10478,"spinlock_wait_cycles":4310912,"thread_start_us":46664,"threads_started":1,"wal-append.queue_time_us":146434,"wal-append.run_cpu_time_us":2276,"wal-append.run_wall_time_us":1447393}
I20260709 13:00:37.095738 11050 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.649600 11321 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.640609 10903 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:37.234333 11193 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:37.116255 11309 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.699432  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 97575.3us
W20260709 13:00:37.720266  8896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.000s	sys 0.013s
W20260709 13:00:37.720638  8896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.000s	sys 0.013s
I20260709 13:00:37.737274  8745 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:00:37.737919  8745 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 2
I20260709 13:00:37.739418  8745 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:37.739786  8745 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:37.119140 11310 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.631731  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:37.596987 11140 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.604671 11235 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.603981 11246 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:37.501822 11108 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.003s
I20260709 13:00:37.141053 11308 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:37.095391  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.669497 11259 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.769047 11277 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:37.118652 11074 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:37.093997 11021 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:37.653748 11172 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.006s
W20260709 13:00:37.680074  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 385) took 2292 ms (client timeout 1977 ms). Trace:
I20260709 13:00:37.680138  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1947) took 2160 ms. Trace:
W20260709 13:00:37.698791  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 96969.6us
I20260709 13:00:37.682195 11278 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.687127 11324 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:37.719681 11011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.005s	sys 0.009s
I20260709 13:00:37.680545 11288 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:37.675696 11326 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:37.769748 11267 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:00:37.677839 11088 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.689476  9442 rpcz_store.cc:269] 0709 13:00:34.760954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.769349 (+  8395us) service_pool.cc:224] Handling call
0709 13:00:37.455921 (+2686572us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54006a8954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.456083 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078708480}
I20260709 13:00:37.766320 11107 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:37.741397 11284 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.004s
W20260709 13:00:37.739959 11314 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:00:37.725469 11238 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:37.766942  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 135398us
I20260709 13:00:37.813531  8788 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:37.752286  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58492.5us
W20260709 13:00:37.814374 11330 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:00:37.769596  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.767853  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1337) took 1186 ms. Trace:
I20260709 13:00:37.816880  8742 rpcz_store.cc:276] 0709 13:00:36.581167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.581265 (+    98us) service_pool.cc:224] Handling call
0709 13:00:36.581568 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:36.581903 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.581930 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:36.588230 (+  6300us) log.cc:844] Serialized 52 byte log entry
0709 13:00:36.806087 (+217857us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:36.806129 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.806197 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.767651 (+961454us) raft_consensus.cc:1685] finished
0709 13:00:37.767671 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.767799 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4301,"spinlock_wait_cycles":71553280,"thread_start_us":172769,"threads_started":1,"wal-append.queue_time_us":259144}
I20260709 13:00:37.772884  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:37.818559  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45779.7us
I20260709 13:00:37.819665  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:37.820530  8822 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:37.820986  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:37.816241 11260 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.677610 11112 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
W20260709 13:00:37.760432 11332 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:00:37.777885  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.795043 11277 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.780884  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1647) took 2090 ms (client timeout 2274 ms). Trace:
W20260709 13:00:37.860532  8889 rpcz_store.cc:269] 0709 13:00:35.690433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.690571 (+   138us) service_pool.cc:224] Handling call
0709 13:00:37.780734 (+2090163us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b54000488d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.780831 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184133376}
I20260709 13:00:37.781855  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:36.563007 10814 ts_tablet_manager.cc:1469] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Trace:
0709 13:00:18.280691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.747766 (+467075us) service_pool.cc:224] Handling call
0709 13:00:19.154117 (+406351us) ts_tablet_manager.cc:934] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
0709 13:00:25.046328 (+5892211us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:00:26.362689 (+1316361us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:26.362821 (+   132us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:00:26.682339 (+319518us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:00:26.682652 (+   313us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:26.682959 (+   307us) ts_tablet_manager.cc:989] Replica registered.
0709 13:00:26.683021 (+    62us) inbound_call.cc:177] Queueing success response
0709 13:00:30.686848 (+4003827us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:00:34.674764 (+3987916us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:00:34.753176 (+ 78412us) tablet_replica.cc:261] Starting instrumentation
0709 13:00:34.859867 (+106691us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":30019,"mutex_wait_us":35873,"raft.queue_time_us":169627,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":182858,"tablet-copy.queue_time_us":218399,"tablet-download-pool.queue_time_us":151,"tablet-download-pool.run_cpu_time_us":1542,"tablet-download-pool.run_wall_time_us":2455850,"thread_start_us":312424,"threads_started":3}
I20260709 13:00:37.759966 11255 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.767508 11309 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:37.653606  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 518) took 1125 ms. Trace:
I20260709 13:00:37.870368  8891 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.738770 11236 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:00:37.874302  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:37.752900 10903 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:37.789927  8894 rpcz_store.cc:276] 0709 13:00:35.519695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.597267 (+ 77572us) service_pool.cc:224] Handling call
0709 13:00:35.614465 (+ 17198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.880491 (+266026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.880526 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:35.881662 (+  1136us) log.cc:844] Serialized 26 byte log entry
0709 13:00:36.486229 (+604567us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:36.546000 (+ 59771us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.546257 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.546355 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.679884 (+1133529us) raft_consensus.cc:1685] finished
0709 13:00:37.679911 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.680059 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65877,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2534,"raft.queue_time_us":61130,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":10480,"thread_start_us":584678,"threads_started":2,"wal-append.queue_time_us":531208}
I20260709 13:00:37.800279  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.801102  8743 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:37.803054  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:37.806653  8787 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.274s (SENT)
W20260709 13:00:37.794490 11011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.006s	sys 0.010s
I20260709 13:00:37.821563  8819 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.772984  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:37.879447  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 72877.3us
I20260709 13:00:37.783711 11108 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.824963 11253 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:37.825167  8741 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.825930  8742 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:00:37.853461  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58164.3us
I20260709 13:00:37.894762  8742 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:37.801138 11324 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.897871 11324 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.898277 11108 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:37.811571  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 517) took 1283 ms. Trace:
I20260709 13:00:37.898389 11324 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:37.898813  9445 rpcz_store.cc:276] 0709 13:00:36.528230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.079938 (+551708us) service_pool.cc:224] Handling call
0709 13:00:37.807350 (+727412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.899587  9443 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:37.899848  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:37.858359  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 3.
W20260709 13:00:37.767824 11251 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.000s
I20260709 13:00:37.800400 11278 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.900698 11324 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:37.863749  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:37.903290  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:37.829798 11172 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:37.872738  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:37.873016 11235 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:37.873536  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:37.870903 11005 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:37.789783  8821 rpcz_store.cc:269] 0709 13:00:35.387287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.423587 (+ 36300us) service_pool.cc:224] Handling call
0709 13:00:37.679897 (+2256310us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400076f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.680009 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348275072}
I20260709 13:00:37.877346  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:37.896109 11271 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.883185 11236 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.909960  9445 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:37.914991 11236 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:37.915321  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 528) took 1570 ms. Trace:
I20260709 13:00:37.915735  9445 rpcz_store.cc:276] 0709 13:00:36.344702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.899582 (+1554880us) service_pool.cc:224] Handling call
0709 13:00:37.915264 (+ 15682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.915975 11236 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.916786  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.822219 10937 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:00:37.887701  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:37.917460 11251 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:37.917897  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 591) took 1029 ms. Trace:
I20260709 13:00:37.917995 11251 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:37.918265  9440 rpcz_store.cc:276] 0709 13:00:36.888499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.887427 (+998928us) service_pool.cc:224] Handling call
0709 13:00:37.917832 (+ 30405us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:37.919627  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 516) took 1674 ms (client timeout 1919 ms). Trace:
W20260709 13:00:37.920019  9449 rpcz_store.cc:269] 0709 13:00:36.245223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.250659 (+  5436us) service_pool.cc:224] Handling call
0709 13:00:37.913086 (+1662427us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.919569 (+  6483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544713472}
I20260709 13:00:37.789743 11259 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.888797  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1650) took 1304 ms. Trace:
I20260709 13:00:37.921233  8745 rpcz_store.cc:276] 0709 13:00:36.584007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.608419 (+1024412us) service_pool.cc:224] Handling call
0709 13:00:37.608789 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.740264 (+131475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.888557 (+148293us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b54001d9a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:37.888744 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108447104}
I20260709 13:00:37.886788  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.922353  8824 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:37.886173  9446 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:37.922715  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1958) took 2595 ms (client timeout 2948 ms). Trace:
I20260709 13:00:37.923352  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 521) took 1394 ms. Trace:
I20260709 13:00:37.923213  8815 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:37.923801  9446 rpcz_store.cc:276] 0709 13:00:36.532290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.741269 (+1208979us) service_pool.cc:224] Handling call
0709 13:00:37.923298 (+182029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.878752  8816 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:37.858387  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:37.851171  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1651) took 2006 ms. Trace:
I20260709 13:00:37.861397  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:37.894199  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:37.803865 11288 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.885433 11331 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:00:37.906519 11289 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:37.885207 11329 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:00:37.911608  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:37.911947  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 98524.5us
I20260709 13:00:37.786178 11308 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.923382  8820 rpcz_store.cc:269] 0709 13:00:35.327043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.377448 (+ 50405us) service_pool.cc:224] Handling call
0709 13:00:37.922540 (+2545092us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:37.922667 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1954679040}
I20260709 13:00:37.922683  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:37.874096 11341 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:37.927899  8819 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:37.928992  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 526) took 1335 ms. Trace:
I20260709 13:00:37.917193 11236 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.928941  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:37.965373  8895 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.929724  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:37.930806 11194 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:37.893534 11253 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:37.931620 11288 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:37.908260 11260 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:00:37.910828 11172 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:37.907665 11338 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.870700  9444 rpcz_store.cc:276] 0709 13:00:36.528509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.585814 (+ 57305us) service_pool.cc:224] Handling call
0709 13:00:37.640105 (+1054291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.886983 11337 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:37.924950  8892 rpcz_store.cc:276] 0709 13:00:35.844719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.221578 (+376859us) service_pool.cc:224] Handling call
0709 13:00:36.244853 (+ 23275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.273418 (+ 28565us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:36.273772 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.573168 (+299396us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.588536 (+ 15368us) log.cc:844] Serialized 26 byte log entry
0709 13:00:36.588741 (+   205us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:36.588785 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.588852 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:36.819733 (+230881us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b48001d01d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:00:37.816434 (+996701us) raft_consensus.cc:1685] finished
0709 13:00:37.829999 (+ 13565us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.851106 (+ 21107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30392,"prepare.queue_time_us":1273,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":32880,"raft.run_cpu_time_us":2028,"raft.run_wall_time_us":245606,"spinlock_wait_cycles":5710720,"thread_start_us":296567,"threads_started":1,"wal-append.queue_time_us":362408}
I20260709 13:00:37.935047 11173 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:37.959093  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:37.922812 11251 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:37.963428 11339 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:37.963853 11326 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:37.966826  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55327.2us
I20260709 13:00:38.006857 11311 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.007401 11311 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:37.966925  8894 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:37.929076 11021 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.938355 10903 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:37.968870  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 532) took 1172 ms. Trace:
I20260709 13:00:38.008285 11311 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.012717  9445 rpcz_store.cc:276] 0709 13:00:36.795066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.916522 (+1121456us) service_pool.cc:224] Handling call
0709 13:00:37.967408 (+ 50886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.013777  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.014617  9445 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:38.015483  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:37.969269  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.938220 11289 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:37.975633  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1336) took 1394 ms. Trace:
I20260709 13:00:38.016327  8820 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:38.017367  8748 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:38.016548  8746 rpcz_store.cc:276] 0709 13:00:36.580816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.580919 (+   103us) service_pool.cc:224] Handling call
0709 13:00:36.622518 (+ 41599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.652261 (+ 29743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.652305 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.652736 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.089336 (+436600us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:37.089381 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.404716 (+315335us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b54003bfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:37.404779 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:37.970840 (+566061us) raft_consensus.cc:1685] finished
0709 13:00:37.970858 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.975563 (+  4705us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67893120,"thread_start_us":341031,"threads_started":1,"wal-append.queue_time_us":922654}
I20260709 13:00:37.976588  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:37.960961  9442 rpcz_store.cc:276] 0709 13:00:36.594298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.819396 (+1225098us) service_pool.cc:224] Handling call
0709 13:00:37.928925 (+109529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.979017 11345 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.980196 11308 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.020535  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.020807  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.021180  9442 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:38.021404  8816 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:38.021543 11277 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.022100  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.022264  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.022933  9442 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:38.023833  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.024111  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 587) took 1213 ms. Trace:
I20260709 13:00:38.024396  8892 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.024528  9442 rpcz_store.cc:276] 0709 13:00:36.810418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.021875 (+1211457us) service_pool.cc:224] Handling call
0709 13:00:38.024035 (+  2160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.983023 11043 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.025400 11173 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.025528  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:37.983832 11290 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:37.984791 11194 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:37.987210  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.026522 11290 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:37.988449  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.025548 11043 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.027364  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 531) took 1250 ms. Trace:
I20260709 13:00:37.982939 11172 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.993124  8895 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:37.972522 11253 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.972193 11264 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:38.026374  9442 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:37.961071 11112 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:38.004303  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.008656  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:38.018488  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1641032569999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:00:38.038486  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1560032570000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:00:38.018739  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 59781.7us
I20260709 13:00:38.000447  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 516) took 1472 ms. Trace:
I20260709 13:00:37.990938 11338 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:37.983140 11336 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:00:38.026968  9446 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:38.027750  9449 rpcz_store.cc:276] 0709 13:00:36.777161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.988209 (+1211048us) service_pool.cc:224] Handling call
0709 13:00:38.027310 (+ 39101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:38.031538  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 121273us
W20260709 13:00:38.001370  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 105313us
I20260709 13:00:38.034535  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 515) took 1789 ms. Trace:
I20260709 13:00:38.032234 11345 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.032209 11260 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:38.026396 11340 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:00:38.013181 11311 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.039542  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.023144 11140 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.041767  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:38.044432  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.044782  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.062242  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:38.066681  8893 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:38.067492  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.030725 10909 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.019171 11339 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.049587 11335 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.002635 11337 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.027472 11290 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.057830  8821 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:00:38.057852  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.059490  8819 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:38.055835 11005 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:38.037127  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.059649  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 590) took 1184 ms. Trace:
I20260709 13:00:38.076516  9449 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:38.077260  9446 rpcz_store.cc:276] 0709 13:00:36.874965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.986955 (+1111990us) service_pool.cc:224] Handling call
0709 13:00:38.059563 (+ 72608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.064621  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.068852  8893 raft_consensus.cc:2368] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6.
I20260709 13:00:38.070734  8892 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:38.058373  9441 rpcz_store.cc:276] 0709 13:00:36.244947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.251779 (+  6832us) service_pool.cc:224] Handling call
0709 13:00:38.034353 (+1782574us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b5400559954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:38.034476 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47823616}
I20260709 13:00:38.073516 10903 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.073446 10909 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.074370 11170 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:38.058444  8908 tablet_copy_service.cc:342] P 7fd110c312c14e17a2bbe6d453493cf2: Request end of tablet copy session 112105225cc140459688be5d31dcb071-fe13736604454be3b391b8e946192961 received from {username='slave'} at 127.0.0.1:40708
I20260709 13:00:38.040490  9440 rpcz_store.cc:276] 0709 13:00:36.527958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.919216 (+1391258us) service_pool.cc:224] Handling call
0709 13:00:37.998876 (+ 79660us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.062776 11296 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.039942 11264 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.171118  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.217954  8893 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:38.218597 11296 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.204061 11342 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.164608 11189 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.072579 11236 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.139s
I20260709 13:00:38.060906 11345 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.178725  8823 raft_consensus.cc:2368] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7.
I20260709 13:00:38.145674  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.180729  8819 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:38.193094  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:38.205133  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 140660us
I20260709 13:00:38.187119  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.183413  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.193389  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.164726 11005 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.208303  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.066308 11260 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.186484  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.220470  8908 tablet_copy_service.cc:434] P 7fd110c312c14e17a2bbe6d453493cf2: ending tablet copy session 112105225cc140459688be5d31dcb071-fe13736604454be3b391b8e946192961 on tablet fe13736604454be3b391b8e946192961 with peer 112105225cc140459688be5d31dcb071
W20260709 13:00:38.225719  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.222636  8820 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:38.267594  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.318055 11140 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.267195  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.318876 10900 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.317399  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 353) took 1440 ms. Trace:
I20260709 13:00:38.178169 11259 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.325562  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 401) took 1647 ms. Trace:
I20260709 13:00:38.326776 11108 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:38.316968 11055 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:38.331059 11296 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.182463 11285 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.076840 11273 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.340481  8816 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:38.289566  8890 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:38.261952  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.360965  8891 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:38.342191  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.341152  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.375000  8818 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.375313  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.378490  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.380909  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.344892  9449 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:38.345335  8821 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:00:38.098659  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.362061  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:38.118757  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.075521 11339 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.321122 11074 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.352118 11189 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.352995  8747 rpcz_store.cc:276] 0709 13:00:36.877376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.622199 (+744823us) service_pool.cc:224] Handling call
0709 13:00:37.668778 (+ 46579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.670552 (+  1774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.670580 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.679987 (+  9407us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.783483 (+103496us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:37.783521 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.783586 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.220099 (+436513us) raft_consensus.cc:1685] finished
0709 13:00:38.220117 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.317332 (+ 97215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1081,"raft.queue_time_us":9558,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":80,"thread_start_us":102784,"threads_started":1,"wal-append.queue_time_us":151779}
W20260709 13:00:38.213567 11318 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.379s	user 0.005s	sys 0.000s
W20260709 13:00:38.355576  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88510.6us
W20260709 13:00:38.356173 11321 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.000s
I20260709 13:00:38.388819  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.350324  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.390198  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.355890  8896 rpcz_store.cc:276] 0709 13:00:36.678217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.884168 (+205951us) service_pool.cc:224] Handling call
0709 13:00:36.940583 (+ 56415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.067942 (+127359us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:37.068611 (+   669us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.721111 (+652500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.721166 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:37.721228 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.325330 (+604102us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:38.325374 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.325501 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":123,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":16014,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":28800,"spinlock_wait_cycles":288594688,"thread_start_us":652137,"threads_started":1,"wal-append.queue_time_us":738861}
W20260709 13:00:38.289477 11336 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:00:38.065642 11326 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.207847 11309 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.341747  8892 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:38.362525 11140 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.358290 11183 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.364213 11055 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:38.364729  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 589) took 1497 ms. Trace:
I20260709 13:00:38.366124  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.350641 11248 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.367650  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1305) took 4215 ms. Trace:
I20260709 13:00:38.369050  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.369966 11005 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.370625  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1323) took 2736 ms. Trace:
I20260709 13:00:38.372825 11270 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:38.384181 11246 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.600s	user 0.004s	sys 0.005s
I20260709 13:00:38.429575  8815 rpcz_store.cc:276] 0709 13:00:34.152169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.609029 (+1456860us) service_pool.cc:224] Handling call
0709 13:00:37.923048 (+2314019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.923893 (+   845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.367457 (+443564us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b5400074c50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:38.367591 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5300237696}
I20260709 13:00:38.410351 11342 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.391201  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:38.374513 11273 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.416004  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.359156 11108 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:38.469411  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.417414  9446 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:38.395145  9412 leader_election.cc:400] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6.
I20260709 13:00:38.385771 11296 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.425109  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.388576 10937 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.392904 11346 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:38.422287 11338 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.374382 11172 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.418793 11140 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.409701 11168 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.442015  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.480506  8898 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:38.481159  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.481863  8898 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:38.482542  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.482897  8898 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.382897  8891 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:38.473579  9412 leader_election.cc:403] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:38.474582  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.476369  8893 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:38.394119 11321 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.440039 11270 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:38.484540  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89610.9us
W20260709 13:00:38.485476 11350 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
I20260709 13:00:38.483522 11346 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:38.492358 11172 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.424304  9442 rpcz_store.cc:276] 0709 13:00:36.867534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.068416 (+1200882us) service_pool.cc:224] Handling call
0709 13:00:38.364662 (+296246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.492792 11346 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.493923  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.495395 11189 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.496047 11260 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.419076 11309 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.474172  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.429634 11253 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.497591 11339 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.497972 11253 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.460003 11246 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.498518 11270 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:38.499451 11263 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.499673 11339 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.499691 11270 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.499977 11246 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.481383 11140 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.459271 11273 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.497993 11309 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.505064 11043 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.507939  9441 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:38.508700 11278 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.427620 11183 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.494041 11346 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.512753  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.456863 11248 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.515803 11337 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.513393 11055 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.529671 11321 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.535049 11345 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:38.425625  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 354403us
I20260709 13:00:38.438094  8743 rpcz_store.cc:276] 0709 13:00:35.634584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.192061 (+557477us) service_pool.cc:224] Handling call
0709 13:00:36.192369 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.879125 (+1686756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.370390 (+491265us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001d8b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:38.370561 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3701172480}
W20260709 13:00:38.458464  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67359.1us
I20260709 13:00:38.572259  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.568660  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.576884  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.424855  8818 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:38.583142  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1308) took 4430 ms. Trace:
I20260709 13:00:38.583741  8815 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:38.583628  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.584914  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.585322  8891 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.585836  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.586228  8819 raft_consensus.cc:2368] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5.
I20260709 13:00:38.587391  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.587430  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.588323  8815 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.589550  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:38.487857  8893 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:38.595188  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.598085  8897 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:38.598302  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.599083  8824 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.601924  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.603559  8815 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:38.603734  8898 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:38.604988  8821 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:38.605456  8821 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:38.605790  8821 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:38.606156  8821 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:38.607975  8821 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:38.613186  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:38.622677  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.597082  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:38.624269  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.489465  9440 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:38.624915  8893 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.626165  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 531) took 1036 ms. Trace:
I20260709 13:00:38.626859  9440 rpcz_store.cc:276] 0709 13:00:37.589726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.268117 (+678391us) service_pool.cc:224] Handling call
0709 13:00:38.625870 (+357753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.508852  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 523) took 1963 ms. Trace:
I20260709 13:00:38.629127  9441 rpcz_store.cc:276] 0709 13:00:36.544931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.421267 (+1876336us) service_pool.cc:224] Handling call
0709 13:00:38.508391 (+ 87124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:38.632622  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.632658  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.634300  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 519) took 2105 ms. Trace:
I20260709 13:00:38.634793  9440 rpcz_store.cc:276] 0709 13:00:36.528819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.629806 (+2100987us) service_pool.cc:224] Handling call
0709 13:00:38.634198 (+  4392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51456}
I20260709 13:00:38.636351  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.637172  9440 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.638844 10909 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.631511  9445 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:38.640086 10937 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.583834  8817 rpcz_store.cc:276] 0709 13:00:34.152887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.752892 (+1600005us) service_pool.cc:224] Handling call
0709 13:00:35.827668 (+ 74776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.850877 (+ 23209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:37.250367 (+1399490us) spinlock_profiling.cc:243] Waited 9.62 ms on lock 0x7b18002458c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:00:37.561939 (+311572us) log.cc:844] Serialized 14 byte log entry
0709 13:00:37.862676 (+300737us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.869455 (+  6779us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.869595 (+   140us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:37.869629 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.869746 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.581916 (+712170us) raft_consensus.cc:1685] finished
0709 13:00:38.582438 (+   522us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.582878 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22770,"prepare.queue_time_us":1723327,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":57804,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":10278,"spinlock_wait_cycles":22120320,"thread_start_us":611566,"threads_started":3,"wal-append.queue_time_us":257942}
I20260709 13:00:38.581089  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.588965  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.658299  8891 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:38.666923  8898 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:38.692119  8816 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:38.700541 11246 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:38.701130 11246 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:38.701611 11246 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:38.705842  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 77217.8us
W20260709 13:00:38.681120  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 100390us
I20260709 13:00:38.702688 11183 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.717808 11074 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.718710 11088 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.719177 11088 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:38.719604 11088 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.722837 11345 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.702137 11321 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:38.727560 11321 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.702822 10968 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.737349 11011 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:38.737874 10968 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.699354 10909 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:38.723985 11248 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.517457 11108 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.742865 11108 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.705911 11173 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.744848 11173 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.737082  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1648) took 2153 ms. Trace:
I20260709 13:00:38.745788  8744 rpcz_store.cc:276] 0709 13:00:36.583443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.603864 (+1020421us) service_pool.cc:224] Handling call
0709 13:00:37.744608 (+140744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.744832 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:37.855910 (+111078us) log.cc:844] Serialized 14 byte log entry
0709 13:00:38.651788 (+795878us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b1800466b40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:00:38.674901 (+ 23113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:38.683002 (+  8101us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:38.707901 (+ 24899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.716051 (+  8150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.736881 (+ 20830us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147480,"prepare.run_cpu_time_us":477,"prepare.run_wall_time_us":7148,"raft.queue_time_us":49814,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":282,"spinlock_wait_cycles":22937856,"thread_start_us":126217,"threads_started":2,"wal-append.queue_time_us":117614}
I20260709 13:00:38.747381 11173 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.702962 11146 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.716184  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.702402 11335 raft_consensus.cc:2764] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:00:38.703064 11005 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.755062 11309 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.755887 11309 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.749238 11146 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:38.761795  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 67558.3us
I20260709 13:00:38.762831 11146 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.763230  8889 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:38.766933 11321 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.767403  8890 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:38.774824  8890 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:38.777572  8897 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:38.778137  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 521) took 2435 ms (client timeout 1840 ms). Trace:
I20260709 13:00:38.778478  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:38.780014  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 513) took 2535 ms (client timeout 1500 ms). Trace:
I20260709 13:00:38.783039  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:38.784550  8788 leader_election.cc:336] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.919s (SENT)
I20260709 13:00:38.787020  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.788690  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:38.789964 11361 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:00:38.789904  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:38.790943  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.791946  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:38.792142  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.800117  8744 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.800719  9441 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:38.801185  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 599) took 1015 ms. Trace:
I20260709 13:00:38.801573  9441 rpcz_store.cc:276] 0709 13:00:37.785949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.630992 (+845043us) service_pool.cc:224] Handling call
0709 13:00:38.791912 (+160920us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:38.801102 (+  9190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14483328}
I20260709 13:00:38.802816  9440 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:38.802879  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.803378  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 601) took 1004 ms. Trace:
I20260709 13:00:38.803730  9440 rpcz_store.cc:276] 0709 13:00:37.799342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.635791 (+836449us) service_pool.cc:224] Handling call
0709 13:00:38.803335 (+167544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.803788  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 534) took 1145 ms. Trace:
I20260709 13:00:38.804309  9441 rpcz_store.cc:276] 0709 13:00:37.658125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.802466 (+1144341us) service_pool.cc:224] Handling call
0709 13:00:38.803719 (+  1253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.804637  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.793890  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:38.799353  8819 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:38.806387  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.807722  8744 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:38.807871  9441 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:38.806411  8816 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:38.809228  9440 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:38.798180  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:38.795996  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 530) took 1206 ms. Trace:
I20260709 13:00:38.812915 11005 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:38.813257  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44162.8us
W20260709 13:00:38.797822  9447 rpcz_store.cc:269] 0709 13:00:36.244339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.251594 (+  7255us) service_pool.cc:224] Handling call
0709 13:00:38.779405 (+2527811us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:38.779738 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3110694784}
I20260709 13:00:38.813907  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 539) took 1152 ms. Trace:
I20260709 13:00:38.813403  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 538) took 1152 ms. Trace:
I20260709 13:00:38.814666  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.814428  9441 rpcz_store.cc:276] 0709 13:00:37.661313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.804958 (+1143645us) service_pool.cc:224] Handling call
0709 13:00:38.813833 (+  8875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.814860  9446 rpcz_store.cc:276] 0709 13:00:37.661026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.629738 (+968712us) service_pool.cc:224] Handling call
0709 13:00:38.790675 (+160937us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:38.813275 (+ 22600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16324608}
I20260709 13:00:38.815872  9447 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:38.816124  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 356) took 1719 ms. Trace:
I20260709 13:00:38.816250  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.817205  9441 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:38.816334  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 547) took 1134 ms. Trace:
I20260709 13:00:38.814496  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:38.817672  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 540) took 1156 ms. Trace:
I20260709 13:00:38.817783  9447 rpcz_store.cc:276] 0709 13:00:37.681708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.814365 (+1132657us) service_pool.cc:224] Handling call
0709 13:00:38.816275 (+  1910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.817997  8824 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:38.819010  9441 rpcz_store.cc:276] 0709 13:00:37.661591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.815939 (+1154348us) service_pool.cc:224] Handling call
0709 13:00:38.817626 (+  1687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.816345  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.826259  8898 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:38.797348  9443 rpcz_store.cc:269] 0709 13:00:36.342814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.574143 (+231329us) service_pool.cc:224] Handling call
0709 13:00:38.777447 (+2203304us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400660154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:38.777816 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929262848}
I20260709 13:00:38.838027  9449 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:38.816541  8748 rpcz_store.cc:276] 0709 13:00:37.096881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.610595 (+513714us) service_pool.cc:224] Handling call
0709 13:00:37.610924 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.017788 (+406864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.815657 (+797869us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54001d9554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:38.815936 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934668032}
I20260709 13:00:38.840394  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 600) took 1049 ms. Trace:
I20260709 13:00:38.843171  9446 rpcz_store.cc:276] 0709 13:00:37.790589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.816049 (+1025460us) service_pool.cc:224] Handling call
0709 13:00:38.840334 (+ 24285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.810791 10968 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.844784  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 543) took 1173 ms. Trace:
I20260709 13:00:38.812562  9445 rpcz_store.cc:276] 0709 13:00:37.589471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.015252 (+425781us) service_pool.cc:224] Handling call
0709 13:00:38.795666 (+780414us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:38.795895 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413696896}
I20260709 13:00:38.850382  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.846676  9449 rpcz_store.cc:276] 0709 13:00:37.671254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.460161 (+788907us) service_pool.cc:224] Handling call
0709 13:00:38.782512 (+322351us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:38.844703 (+ 62191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32147200}
I20260709 13:00:38.849189  8897 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:38.855290  8893 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:38.857059  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.839267  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 533) took 1169 ms. Trace:
W20260709 13:00:38.860778  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 76358.9us
W20260709 13:00:38.860431  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46270.4us
I20260709 13:00:38.859802  8890 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:38.859596  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 541) took 1197 ms. Trace:
I20260709 13:00:38.852562 11005 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.873818  9447 rpcz_store.cc:276] 0709 13:00:37.661851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.819342 (+1157491us) service_pool.cc:224] Handling call
0709 13:00:38.859513 (+ 40171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.859573  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.875101  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.877676  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.879238 10937 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6.
I20260709 13:00:38.881446  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.882187  8892 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.884583  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.885742  9441 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:38.886242  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 592) took 1974 ms. Trace:
I20260709 13:00:38.886621  9441 rpcz_store.cc:276] 0709 13:00:36.911637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.840260 (+1928623us) service_pool.cc:224] Handling call
0709 13:00:38.886138 (+ 45878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.886979 11293 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.888872  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:38.889982  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.873922  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:38.891992  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.880535 11289 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.897753  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.867309 11140 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:38.899758  8822 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:38.897792  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.891112  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.901543  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 548) took 1219 ms. Trace:
I20260709 13:00:38.901921  9449 rpcz_store.cc:276] 0709 13:00:37.681970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.897520 (+1215550us) service_pool.cc:224] Handling call
0709 13:00:38.901489 (+  3969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.858795  9440 rpcz_store.cc:276] 0709 13:00:37.657838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.804410 (+1146572us) service_pool.cc:224] Handling call
0709 13:00:38.818180 (+ 13770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.903095  8893 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:38.903218 11140 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.904052 11240 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:38.904139  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.905990  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.906901  8824 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:38.899358  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:38.910971  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.904250  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.857457 11309 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.916653  8890 raft_consensus.cc:2368] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:00:38.895459 11021 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.917524  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.911962  8897 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:38.919253  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.920030  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:38.921319  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.921998  8898 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:38.854250 11273 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:38.922734  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.925133  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.926430  9447 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:38.926771 11326 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:00:38.911676 11285 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:38.929545  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:38.930832  8898 raft_consensus.cc:2368] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7.
I20260709 13:00:38.931216  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.931708  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:38.931808  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:38.936585  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:38.932441  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:38.938045  8816 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.940150  9445 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:38.941776  8893 raft_consensus.cc:2376] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6. Re-sending same reply.
I20260709 13:00:38.943094  8820 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
W20260709 13:00:38.947423  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 79100.5us
I20260709 13:00:38.947888 11349 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:38.948598 11349 consensus_queue.cc:1243] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:38.949056  8864 leader_election.cc:400] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5.
I20260709 13:00:38.949447  8864 leader_election.cc:403] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:38.951682 11264 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:38.951598  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:38.954697  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:38.959179  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.940668  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 550) took 1164 ms. Trace:
I20260709 13:00:38.964280  8818 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:38.967644  8815 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:00:38.974948  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.981801  9441 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:38.982270  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 593) took 2066 ms. Trace:
I20260709 13:00:38.982523  9441 rpcz_store.cc:276] 0709 13:00:36.915752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.901220 (+1985468us) service_pool.cc:224] Handling call
0709 13:00:38.982221 (+ 81001us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.963270  9445 rpcz_store.cc:276] 0709 13:00:37.775815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.898962 (+1123147us) service_pool.cc:224] Handling call
0709 13:00:38.940608 (+ 41646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.997707  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 596) took 1954 ms. Trace:
I20260709 13:00:38.998152  9445 rpcz_store.cc:276] 0709 13:00:37.043386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.996261 (+1952875us) service_pool.cc:224] Handling call
0709 13:00:38.996715 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.997261 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:38.997322 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:38.997370 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:38.997441 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.997507 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.997629 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:39.005774  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64174.5us
I20260709 13:00:39.006476  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:39.007213  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:39.007694  9444 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:39.010589 11183 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.011173 11183 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:39.012210 11183 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.016705 11273 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:39.017544 11289 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.020462 11055 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.026935 10909 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:39.027825 10909 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.031605  9441 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:39.032371  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 588) took 2176 ms. Trace:
I20260709 13:00:39.032719  9441 rpcz_store.cc:276] 0709 13:00:36.856464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.983164 (+2126700us) service_pool.cc:224] Handling call
0709 13:00:39.031260 (+ 48096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.032111 (+   851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.032297 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:39.036075  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.054033  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78158.1us
I20260709 13:00:39.056301  8892 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:39.053897  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:39.008589  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 595) took 2042 ms (client timeout 2203 ms). Trace:
W20260709 13:00:39.078162  9444 rpcz_store.cc:269] 0709 13:00:36.965731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.004780 (+1039049us) service_pool.cc:224] Handling call
0709 13:00:39.006016 (+1001236us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:39.008510 (+  2494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17272192}
I20260709 13:00:39.080214  9444 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: Advancing to term 6
I20260709 13:00:39.075740  8741 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:39.076220 11264 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5.
I20260709 13:00:39.076310  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:39.077368 11326 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:39.095721  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.096478  8746 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:39.101616  8744 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:39.103197  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:39.104027  8744 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:39.027047 11021 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:00:39.109953  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.110739  8743 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:39.111502  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.113216  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:39.113894  8816 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:39.105062  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.212227  8819 raft_consensus.cc:2368] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
W20260709 13:00:39.224308  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.118098  8746 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:39.152143  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70352.8us
I20260709 13:00:39.115538  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.251219  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:39.277889 11372 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:00:39.212481 11349 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:39.282542  8744 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:39.284626  9442 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:39.151132 11273 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:39.282716  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:39.297071  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.297839  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:39.297914  8745 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:39.298744  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.299003  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:39.299571  8744 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:39.299612  8745 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.300602  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.301707  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.302740  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:39.303229  8744 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.307612  8745 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:39.308578  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.233083 10909 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.309413  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 520) took 2951 ms. Trace:
I20260709 13:00:39.310032  8744 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:39.310676 10909 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:39.311506 11246 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.007s
I20260709 13:00:39.311864 11246 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.309798  9440 rpcz_store.cc:276] 0709 13:00:36.342547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.903248 (+2560701us) service_pool.cc:224] Handling call
0709 13:00:38.903769 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.904355 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:38.904693 (+   338us) log.cc:844] Serialized 14 byte log entry
0709 13:00:38.954176 (+ 49483us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.138285 (+184109us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.205277 (+ 66992us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:39.206646 (+  1369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.236389 (+ 29743us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:39.280182 (+ 43793us) raft_consensus.cc:1685] finished
0709 13:00:39.281885 (+  1703us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.292128 (+ 10243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":855,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":9966,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":280,"thread_start_us":27825,"threads_started":1,"wal-append.queue_time_us":243470}
I20260709 13:00:39.314039 11253 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:39.314819 11253 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:39.338028  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55607.7us
I20260709 13:00:39.331873  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 529) took 2623 ms. Trace:
W20260709 13:00:39.343719  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 551) took 1467 ms (client timeout 1768 ms). Trace:
I20260709 13:00:39.342466 11273 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.343689  8889 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:39.345553  8746 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:39.344777  9442 rpcz_store.cc:269] 0709 13:00:37.876109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.493642 (+617533us) service_pool.cc:224] Handling call
0709 13:00:39.343514 (+849872us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b5400740254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:39.343664 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971285888}
I20260709 13:00:39.138374 11088 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.367589  8889 pending_rounds.cc:85] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 1
I20260709 13:00:39.376909  8889 pending_rounds.cc:107] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:39.377238  8889 raft_consensus.cc:2889] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:39.379150  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:39.381393  8742 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:39.390192  8743 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.387682  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:39.344841  9449 rpcz_store.cc:276] 0709 13:00:36.670837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.943846 (+2273009us) service_pool.cc:224] Handling call
0709 13:00:38.974158 (+ 30312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.029974 (+ 55816us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:39.030311 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.293749 (+263438us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:39.293805 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:39.293908 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.293987 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.294092 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4199,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":4468,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":258466,"threads_started":1}
W20260709 13:00:39.402558  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.495973  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 117008us
I20260709 13:00:39.704479  8746 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
W20260709 13:00:39.733388  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.664253  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.519344 11371 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
W20260709 13:00:39.782634  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48798.1us
I20260709 13:00:39.409075 11011 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.408468 11281 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.732954 11335 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:39.792285  9447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.016s	sys 0.046s
I20260709 13:00:39.789264  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:39.730569 11273 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:39.519338 11284 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:39.799404  8747 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:39.801946 11246 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:39.788532 11240 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.007s	sys 0.035s
I20260709 13:00:39.802619 11281 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:39.803242 11240 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.007s	sys 0.036s
I20260709 13:00:39.803398  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 603) took 1313 ms. Trace:
W20260709 13:00:39.804795  9447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.017s	sys 0.046s
I20260709 13:00:39.792564 11088 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:39.398182 11349 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:39.562976 11293 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.003s
W20260709 13:00:39.811033  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 198817us
I20260709 13:00:39.519562 11309 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.285s
I20260709 13:00:39.816231  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 514) took 3285 ms. Trace:
I20260709 13:00:39.824054  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:39.811074 11253 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:39.814975 11284 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:39.803862  9440 rpcz_store.cc:276] 0709 13:00:38.490288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.314454 (+824166us) service_pool.cc:224] Handling call
0709 13:00:39.314820 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.315194 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:39.315777 (+   583us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.800875 (+485098us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.803053 (+  2178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:39.803099 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:39.803151 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.803221 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.803341 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3490,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":76917,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":11329,"spinlock_wait_cycles":11528064,"thread_start_us":391324,"threads_started":1}
I20260709 13:00:39.839607 11043 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.840216 11349 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:39.841435  9448 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:00:39.844043  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:39.850549 10992 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:39.845851 11246 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.855880  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 605) took 1365 ms. Trace:
I20260709 13:00:39.859562  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:39.857174  8713 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.873s (SENT)
I20260709 13:00:39.859370  9442 rpcz_store.cc:276] 0709 13:00:38.490796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.385011 (+894215us) service_pool.cc:224] Handling call
0709 13:00:39.385427 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.385755 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:39.386012 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.855470 (+469458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:39.855543 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:39.855602 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.855704 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.855828 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4655,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3711,"raft.queue_time_us":2408,"thread_start_us":466928,"threads_started":1}
W20260709 13:00:39.858512 11055 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.004s
I20260709 13:00:39.864547 11296 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.875234 11296 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.856029 11043 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:39.861095 10909 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:39.880050  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 437740us
I20260709 13:00:39.849568 11335 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.868930 11246 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:39.880834 10909 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:39.891237  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:39.877485 10992 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.891425 11289 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:39.916204  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60511.5us
I20260709 13:00:39.892951 11240 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.894503 11281 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:39.896492 10909 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.904630 11183 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.943889  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:39.938978  9444 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.858s	user 0.015s	sys 0.000s
I20260709 13:00:39.894150 11264 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:00:39.917927 11289 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:39.947161  8713 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.900s (SENT)
W20260709 13:00:39.950935  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1340) took 3302 ms (client timeout 2203 ms). Trace:
W20260709 13:00:39.955062  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 480533us
W20260709 13:00:39.953903  8749 rpcz_store.cc:269] 0709 13:00:36.582950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.877952 (+295002us) service_pool.cc:224] Handling call
0709 13:00:39.876244 (+2998292us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400358c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:39.885631 (+  9387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3324940160}
W20260709 13:00:39.966591  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78766.3us
I20260709 13:00:39.968887 11326 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:39.969367 11326 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:39.969632 11386 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:00:39.970618 11270 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.970306 11326 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.971055 11270 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:39.971674 11326 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.972081 11270 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.973224 11270 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.954527 11289 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.952766  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 533) took 2362 ms. Trace:
W20260709 13:00:39.975232  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.976173  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:39.954020 11183 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:39.977301  8713 leader_election.cc:336] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.991s (SENT)
W20260709 13:00:39.978185  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.979496 11389 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:00:39.979857  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.979830  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 523) took 3636 ms. Trace:
W20260709 13:00:39.989060 11385 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:00:39.988847 11043 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.977806  9445 rpcz_store.cc:276] 0709 13:00:37.590197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.999383 (+1409186us) service_pool.cc:224] Handling call
0709 13:00:38.999867 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:39.028868 (+ 29001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.028900 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:39.117901 (+ 89001us) log.cc:844] Serialized 52 byte log entry
0709 13:00:39.489604 (+371703us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:39.494391 (+  4787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.518566 (+ 24175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:39.940220 (+421654us) raft_consensus.cc:1685] finished
0709 13:00:39.942454 (+  2234us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.951485 (+  9031us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26058,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":220734,"threads_started":1,"wal-append.queue_time_us":712593}
I20260709 13:00:39.977058 11264 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:39.994351  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 77275.3us
I20260709 13:00:39.917754 11284 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:39.985137  9446 rpcz_store.cc:276] 0709 13:00:36.343331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.916342 (+2573011us) service_pool.cc:224] Handling call
0709 13:00:38.930278 (+ 13936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.957839 (+ 27561us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:39.166302 (+208463us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.788387 (+622085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:39.788433 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:39.794932 (+  6499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.795038 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.979619 (+184581us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b540065f4d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:39.979742 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69818,"prepare.queue_time_us":10818,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":74758,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5649,"spinlock_wait_cycles":6566784,"thread_start_us":613440,"threads_started":2,"wal-append.queue_time_us":649609}
I20260709 13:00:39.957767 11240 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:39.979781 11390 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.996222 11264 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.957813 11246 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.004853 11309 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:40.031677  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1957) took 2165 ms. Trace:
I20260709 13:00:40.031561  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:40.030090  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.032217  8889 rpcz_store.cc:276] 0709 13:00:37.863307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.765093 (+901786us) service_pool.cc:224] Handling call
0709 13:00:38.765524 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.413970 (+648446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.023239 (+609269us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b5400030cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:40.028236 (+  4997us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329304320}
I20260709 13:00:40.039654  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.041900 11392 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:40.042477 11392 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:40.042765 11180 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.042863 11392 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:40.043395 11180 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:40.045997 11387 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:00:40.047314 11383 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:00:40.048516 11384 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:00:40.053786  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.055828  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.056466 11281 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.056972 11281 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.060055 11281 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.032807 11264 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.065804  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:40.066736 11253 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:40.067186 11253 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:40.067354  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.067647 11253 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.069949 11335 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.070858 11335 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:40.071079 10909 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.074331  8891 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.075106  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1663) took 2212 ms. Trace:
W20260709 13:00:40.035533  9415 connection.cc:256] RPC call timeout handler was delayed by -1.495993833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 13:00:40.075493  8891 rpcz_store.cc:276] 0709 13:00:37.862865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.029326 (+2166461us) service_pool.cc:224] Handling call
0709 13:00:40.074098 (+ 44772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.074822 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.075041 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:40.038187  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:40.099474  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 104845us
I20260709 13:00:40.096302  8748 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:40.108443  8815 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:40.108981  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:40.109695  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:40.109855  9415 connection.cc:256] RPC call timeout handler was delayed by -1.3969938330000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
W20260709 13:00:40.107858  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1964) took 2299 ms (client timeout 1593 ms). Trace:
W20260709 13:00:40.111194  8823 rpcz_store.cc:269] 0709 13:00:37.795723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.795840 (+   117us) service_pool.cc:224] Handling call
0709 13:00:40.022698 (+2226858us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540028bc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:40.094844 (+ 72146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688480640}
I20260709 13:00:40.111974  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:40.112444  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:40.112573  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 371) took 1256 ms. Trace:
I20260709 13:00:40.113101  8748 rpcz_store.cc:276] 0709 13:00:38.844916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.028978 (+1184062us) service_pool.cc:224] Handling call
0709 13:00:40.100983 (+ 72005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:40.113333  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:40.113955  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:40.114591  8743 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:40.115852  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:40.116106 11300 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.116460 11300 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.116806 11300 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.117547 10909 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:40.119776  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.121785 11180 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.122553 11180 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.125509  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:40.092964 11281 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.124631 11296 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:40.170886  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.171211  8746 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:40.154286  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 212033us
I20260709 13:00:39.822021  9447 rpcz_store.cc:276] 0709 13:00:36.527336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.927293 (+2399957us) service_pool.cc:224] Handling call
0709 13:00:38.927802 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.928215 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:38.928617 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.812078 (+883461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:39.812195 (+   117us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:39.812281 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.812412 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.812674 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4710,"prepare.queue_time_us":917,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":13633,"raft.queue_time_us":6015,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":3039,"thread_start_us":876951,"threads_started":1}
I20260709 13:00:40.177356  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:40.179903  8819 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:40.174291  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:40.182785  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 240158us
I20260709 13:00:35.812211  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:40.157408 11281 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:40.185508  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 296543us
I20260709 13:00:40.179035 11043 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.184554  8823 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:40.188230  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.186523  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 74323.5us
W20260709 13:00:40.188869  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 142524us
I20260709 13:00:40.178158 10992 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:40.219619  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 142450us
I20260709 13:00:40.186364 11296 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.233059  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:40.235304  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.240851  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.190265 11281 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:40.246330  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 179172us
W20260709 13:00:40.249007  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 68705us
I20260709 13:00:40.252013  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:40.247017  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.255587 11293 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:40.277484 11397 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:40.277940 11293 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.278384 11293 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.278266 11397 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.280781 11397 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:40.281695 11398 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:00:40.237572 11055 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.282341  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52543.4us
I20260709 13:00:40.281451  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.288070  9447 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:40.232803 10992 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:40.237355 11273 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:40.221540 11043 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.281454 11246 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:40.301069  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.301863  9446 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:40.277261 11409 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:40.303890  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.290043 11397 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.293352 11183 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.290432 11414 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.308882  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.310237 11406 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.310837 11406 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.313640 11290 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.317483  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.318163 11253 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.318646 11402 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:00:40.319744 11402 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:40.320079 11402 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:40.320291 11253 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.321184  8744 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.321830 11246 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:40.322788 11246 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.324252 11300 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.292972 11055 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.326200 11088 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:40.326534  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1363) took 1026 ms. Trace:
I20260709 13:00:40.326588 11088 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:40.327064 11055 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.327811 11240 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.327646 11088 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.329020 11088 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.284065 11011 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:00:40.333369  9447 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:40.335459  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.329370 11414 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.313099 11406 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.296587 11273 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.345752  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.253645 11296 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.307044 11409 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:40.361505  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.297801  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 602) took 2492 ms. Trace:
I20260709 13:00:40.327065  8742 rpcz_store.cc:276] 0709 13:00:39.299960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.382448 (+ 82488us) service_pool.cc:224] Handling call
0709 13:00:39.382919 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.383422 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:39.784078 (+400656us) log.cc:844] Serialized 14 byte log entry
0709 13:00:39.945084 (+161006us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.966703 (+ 21619us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.966892 (+   189us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:39.966939 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.967033 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.326340 (+359307us) raft_consensus.cc:1685] finished
0709 13:00:40.326359 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.326481 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10166,"prepare.queue_time_us":402664,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":47,"raft.queue_time_us":8791,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":77599,"thread_start_us":127588,"threads_started":1,"wal-append.queue_time_us":207835}
I20260709 13:00:40.359216  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.360112  9442 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:00:40.399771  9442 pending_rounds.cc:85] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 5
I20260709 13:00:40.400051  9442 pending_rounds.cc:107] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:00:40.400259  9442 raft_consensus.cc:2889] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:40.300140 10992 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.366428 11309 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:40.367177  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58600.9us
I20260709 13:00:40.367949 11407 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:40.369501  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.408213  9447 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:40.340502 11414 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.374361 11043 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.343447 11240 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:40.374816 11409 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.320932 11402 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.347033 11264 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.388399  9443 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.388762  8893 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:40.391623 11281 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.393194 11392 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.394675 11300 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.395448 11326 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.397020  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:40.397954  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50018.4us
W20260709 13:00:40.404910  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.330241 11397 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.406762 11407 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:40.411962  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.438794  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.412285 11290 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.411777 11412 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.446956 11415 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:40.447683 11415 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.449431 11413 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.449704  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.450503  9449 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:40.451462 10992 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.451748  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.452517  9449 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:40.455511  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.421299 11240 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:40.463132  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.464226  9446 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.455546  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1362) took 1155 ms. Trace:
W20260709 13:00:40.468302  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 231927us
W20260709 13:00:40.462339  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 169236us
I20260709 13:00:40.439905 11407 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:40.465181  8745 rpcz_store.cc:276] 0709 13:00:39.299670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.317458 (+ 17788us) service_pool.cc:224] Handling call
0709 13:00:39.783221 (+465763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.806468 (+ 23247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:39.942157 (+135689us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.283087 (+340930us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.455118 (+172031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:40.455186 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:40.455250 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.455335 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.455476 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54456,"prepare.queue_time_us":34562,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":254880,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":30427,"thread_start_us":604872,"threads_started":4,"wal-append.queue_time_us":425528}
I20260709 13:00:40.480443  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.483405  9442 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:00:40.502806  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.458840 11088 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.437753 11326 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:40.506258  9448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 522) took 3973 ms (client timeout 1874 ms). Trace:
W20260709 13:00:40.518416  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 272528us
W20260709 13:00:40.518697  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.519300  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.520696  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.521564  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.525444  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1364) took 1161 ms. Trace:
W20260709 13:00:40.526338  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 290005us
I20260709 13:00:40.525856 11240 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.525835  8740 rpcz_store.cc:276] 0709 13:00:39.300249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.470430 (+170181us) service_pool.cc:224] Handling call
0709 13:00:39.525024 (+ 54594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.525228 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:39.993693 (+468465us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.241656 (+247963us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.461079 (+219423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:40.461146 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:40.461206 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.461316 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.461465 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":304148,"prepare.queue_time_us":287070,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1779,"raft.queue_time_us":255978,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":6471,"spinlock_wait_cycles":21538688,"thread_start_us":608661,"threads_started":4,"wal-append.queue_time_us":267474}
I20260709 13:00:40.527532  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:40.528422  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:40.530126 11397 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.531829 11418 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:40.532372 11418 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.532765 11418 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.533456  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:40.533648 11412 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:40.534694  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:40.533200  9443 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:40.517069  9448 rpcz_store.cc:269] 0709 13:00:36.532598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.660546 (+127948us) service_pool.cc:224] Handling call
0709 13:00:40.485161 (+3824615us) spinlock_profiling.cc:243] Waited 3.15 s on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:40.506180 (+ 21019us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7249228544}
I20260709 13:00:40.542521  9442 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:40.457866 11290 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.481391 11402 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.516067  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:40.520735  9412 leader_election.cc:336] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.511s (SENT)
W20260709 13:00:40.542938  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 106571us
I20260709 13:00:40.552016  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:40.552929  8891 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:40.546394  9449 raft_consensus.cc:2376] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6. Re-sending same reply.
I20260709 13:00:40.559469  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:40.543833  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 134588us
I20260709 13:00:40.560338  8889 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:40.563755  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.564532  9448 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:40.526098 11390 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.574085  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:40.574295  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53746.1us
I20260709 13:00:40.574828  8891 raft_consensus.cc:2376] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5. Re-sending same reply.
I20260709 13:00:40.574896  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.575702 11417 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:00:40.568003 11416 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:40.577770 11416 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:40.578217  8712 leader_election.cc:400] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:00:40.578176 11416 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.578881 11236 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.578536  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:40.579443 11236 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.580971 11264 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:40.581605  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60941.8us
I20260709 13:00:40.584523 11425 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:40.586033 11296 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.586529  8895 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.586648  8892 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 6. Ops: []
I20260709 13:00:40.587307 11296 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:40.587695  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:40.588336  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:40.588995  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:40.589756  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:40.590485  8896 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:40.590545  8894 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:40.578671  8712 leader_election.cc:403] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:00:40.558224 11270 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.000s
I20260709 13:00:40.589798  8889 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:40.596378  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.597224  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.597747 11415 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.598366 11427 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.598452 11290 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.599121 11430 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:40.599542  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:40.600024 11413 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.600678 11392 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.599998 11416 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.600848 11428 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:40.602276  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:40.602914 11246 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:40.568871  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 159286us
W20260709 13:00:40.607363  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72780.6us
I20260709 13:00:40.608871  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:40.557752 11412 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:40.566265 11309 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:40.632187  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:40.642356  8889 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:40.643086  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 128487us
I20260709 13:00:40.605789  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.644480 11246 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.645483 11417 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:40.645936  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.646359 11415 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:40.647240 11390 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.647377  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.648147 11415 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:40.648612  9440 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:40.649106 11415 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.649657  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.650262  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:40.651090  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.652105  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.646044 11417 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:40.652845  9440 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:40.653946 11412 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.653172 11417 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.655151  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.656263  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.657198  9440 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:40.657435 11309 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:40.657531 11419 raft_consensus.cc:2764] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:40.658473  9440 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:40.658588  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:40.660908  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61469.6us
I20260709 13:00:40.663606  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.668968  8712 leader_election.cc:400] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7.
I20260709 13:00:40.669428  8712 leader_election.cc:403] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:00:40.675513  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.677412 11300 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.677904 11300 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.680204 11300 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.680644 11300 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.681548 11281 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.681833  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.682242 11281 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.682614 11281 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.682889  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.682979 11281 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.683972  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:40.685609  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:40.686444 11289 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.000s
I20260709 13:00:40.686410 11390 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:40.687194  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.688201 11392 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.688714  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.672079 11421 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.690374 11326 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:40.672765  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64263.2us
I20260709 13:00:40.687001 11289 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.658551 11309 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.696019  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:40.668035  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 89772us
I20260709 13:00:40.398778  9444 rpcz_store.cc:276] 0709 13:00:37.805030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.079367 (+1274337us) service_pool.cc:224] Handling call
0709 13:00:39.079921 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.995799 (+915878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:40.005437 (+  9638us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:40.072422 (+ 66985us) log.cc:844] Serialized 26 byte log entry
0709 13:00:40.241565 (+169143us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:40.253302 (+ 11737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.267980 (+ 14678us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.284760 (+ 16780us) raft_consensus.cc:1685] finished
0709 13:00:40.290154 (+  5394us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.295761 (+  5607us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9716,"raft.queue_time_us":6322,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":10505,"thread_start_us":159128,"threads_started":1,"wal-append.queue_time_us":179465}
I20260709 13:00:40.606381 11293 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.705155  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:40.705916  8889 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:40.640532 11296 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:00:40.712945  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51203.1us
I20260709 13:00:40.713822  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:40.681079 11300 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.712014 11390 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:00:40.731925  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.724351 11435 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:40.734503  8818 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:40.735391  8816 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:40.736582  8821 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:00:40.737371  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.738329  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.739910 11407 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:40.741398  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52793us
I20260709 13:00:40.707144 11421 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:00:40.771252  8817 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:40.744192 11281 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.746891  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:40.764755  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 123220us
I20260709 13:00:40.746295  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.779650  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.780228  8819 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:40.707310 11425 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.659337 11426 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:00:40.745370 11419 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.774262  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.725910 11416 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:40.627559 11427 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:40.779422  8823 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:40.782984  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:40.729746 11290 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.776655  8715 leader_election.cc:400] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:00:40.783087  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.731531 11293 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.791604  8822 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:40.766616 11435 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:40.797905  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:40.726115 11431 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:00:40.796627 11414 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.856278 11407 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:41.246941  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 446055us
W20260709 13:00:41.023360  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.247882  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.248066 11414 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.789878 11326 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.463s
I20260709 13:00:40.852131 11011 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.397s
I20260709 13:00:40.790696 11390 consensus_queue.cc:1243] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:00:40.843624 11430 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:41.258126 11430 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:41.259114 11430 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:41.259899  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 425075us
I20260709 13:00:41.260999  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:40.852370 11281 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:40.994626  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.803879  8823 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.264732  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:40.854262 11427 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:41.265652 11427 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.266239 11427 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.268323 11419 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7.
W20260709 13:00:41.249716  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 554052us
W20260709 13:00:41.270380 11413 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.658s	user 0.006s	sys 0.004s
I20260709 13:00:41.270812 11413 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
W20260709 13:00:40.818579  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104908us
I20260709 13:00:41.271652  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:40.854172 11409 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:41.276348 11409 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:40.789695 11270 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.462s
I20260709 13:00:40.980643 11426 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:40.855357  9444 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:41.282644 11428 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.282534 11426 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.283128 11428 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.283448  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:40.855553 11088 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:41.252549  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.252617  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:41.244751  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 468530us
I20260709 13:00:40.859372 11418 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:40.819312 11435 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:41.263695 11402 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:41.270592  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:40.866999 11289 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:40.980894 11290 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:41.278091 11446 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:40.929563  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49838us
I20260709 13:00:41.278100  8823 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:41.049605  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 549) took 3360 ms. Trace:
I20260709 13:00:40.835978 11284 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:40.881362  9446 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:40.860553 11392 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:41.311169 11296 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:41.313413  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:41.321811  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57225.2us
I20260709 13:00:41.307816 11427 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:41.327797  8819 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:41.333802 11270 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:41.339280  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.339779  8715 leader_election.cc:400] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7.
W20260709 13:00:41.342921  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71429.2us
W20260709 13:00:41.343075  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72641us
I20260709 13:00:41.327312 11183 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.346113  9440 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:41.322186 11437 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:00:41.348521 11290 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.389876 11426 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:41.344578  8820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.010s	sys 0.022s
I20260709 13:00:41.319216 11407 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.390461 11430 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.390599 11407 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.390992 11407 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.391021 11430 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.386739 11440 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:41.388070 11183 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.394701 11418 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.345377 11397 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.396010 11397 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:41.347651 11309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.011s	sys 0.018s
W20260709 13:00:41.397279 11309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.012s	sys 0.019s
I20260709 13:00:41.375279 11402 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:41.398625 11402 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.388252  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:41.345453 11409 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:41.348489 11281 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:41.345917 11284 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.375049 11446 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:41.402805  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.403098  8820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.012s	sys 0.031s
W20260709 13:00:41.363371  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.403334  8819 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:41.344388  9441 rpcz_store.cc:276] 0709 13:00:37.689352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.055907 (+1366555us) service_pool.cc:224] Handling call
0709 13:00:40.502758 (+1446851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.637787 (+135029us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:40.637883 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:40.642689 (+  4806us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:40.667226 (+ 24537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.667328 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.049347 (+382019us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400430b50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:41.049522 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83528,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":3194635136,"thread_start_us":112241,"threads_started":1}
I20260709 13:00:41.343899  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:41.333276 11326 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.383299  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:41.375067 11296 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:41.409615  9441 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:41.413486  8863 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.203s (SENT)
I20260709 13:00:41.415832  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.402386 11236 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.396034 11183 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:41.378048 11270 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:41.419021 11270 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.403764 11430 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:41.417110 11326 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.405791 11284 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:41.420307  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 85868.3us
I20260709 13:00:41.418105  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:41.412184 11440 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:41.426012  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 536) took 2930 ms (client timeout 1834 ms). Trace:
W20260709 13:00:41.442777  9443 rpcz_store.cc:269] 0709 13:00:38.495239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.852056 (+356817us) service_pool.cc:224] Handling call
0709 13:00:41.425836 (+2573780us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54006a95d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:41.425937 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3474142080}
W20260709 13:00:41.426818  8893 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1681) took 2471 ms (client timeout 2058 ms). Trace:
I20260709 13:00:41.426749 11326 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:41.429193 11446 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.413147 11421 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:00:41.429486 11430 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.448017  8816 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:41.448372 11430 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:41.412415 11281 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:41.449218  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.450058 11281 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.450408 11290 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.450461  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.451134  8816 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
W20260709 13:00:41.444862  8893 rpcz_store.cc:269] 0709 13:00:38.958069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.125184 (+1167115us) service_pool.cc:224] Handling call
0709 13:00:41.426578 (+1301394us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400249854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:41.426708 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603093248}
I20260709 13:00:41.453269 11183 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:41.453624 11430 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.434382  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.434762  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:41.457026  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:41.444407  9443 pending_rounds.cc:85] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 7
I20260709 13:00:41.435562  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.457753  9449 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:41.458146  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 614) took 1026 ms. Trace:
I20260709 13:00:41.458174  8824 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:41.458922  9443 pending_rounds.cc:107] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:00:41.436041 11436 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.436980  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.435364 10992 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.458527  9449 rpcz_store.cc:276] 0709 13:00:40.436544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.558743 (+122199us) service_pool.cc:224] Handling call
0709 13:00:41.456883 (+898140us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:41.458095 (+  1212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66328320}
I20260709 13:00:41.459362  9443 raft_consensus.cc:2889] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:41.419274 11296 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:41.463066  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74968.4us
I20260709 13:00:41.452052  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.482071  8816 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 4: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:41.487180  9448 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:41.496102  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 248392us
I20260709 13:00:41.502689  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.514947  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:41.456509 11289 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:41.514986  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:41.515769  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:41.519526  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:41.519774  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.515290  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:41.514990  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.519443  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.521555  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:41.520423  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.522286  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.522711  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:41.522904  8741 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.519702  8898 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:41.520129  8889 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:41.524914  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:41.524961  9449 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:41.520329  8742 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8. Re-sending same reply.
I20260709 13:00:41.530818 11281 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:41.531417 11281 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.531872 11281 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.533931  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.532747  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.535339  8893 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:41.492960 11293 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.533037  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.488831 11326 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:41.538527  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:41.539090  9444 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:41.502523 10992 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.539707  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:41.553099  9448 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.539783  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.533568  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:41.525812  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.520946  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:41.540787  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.541016  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.542042  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.554307  8894 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:41.494470 11183 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.545854  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.546427  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.547488  8892 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.547644  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.548566  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:41.549875  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.555677  8746 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:41.550437  9449 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:41.556002  8742 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:41.551011  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.557964  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:41.559087  8897 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:41.562037  8896 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:41.563633  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:41.562988  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.570638  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.571560  8745 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:41.509704 11426 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:00:41.525497 11273 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.534662 11055 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:41.595326  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.558712 11449 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.601991  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:41.600526  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80529.9us
I20260709 13:00:41.582458  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.583457  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.537268 10909 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:00:41.590924  8893 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.511492 11429 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:00:41.502174 11436 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.510362 11407 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.602190  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.636839  8744 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 3.
I20260709 13:00:41.602568  8746 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.603121  8741 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:41.585016  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:41.581488  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66649.5us
I20260709 13:00:41.563386 11450 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.638255  8747 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.638541 11450 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.639074 11450 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:41.624924  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 84241.9us
I20260709 13:00:41.644646  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.627580  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.611003  8892 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:41.552662 11293 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.633314  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:41.645709  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.645964  8823 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:41.645725  8747 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:41.647814  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.564121 11183 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.667042  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.596647 11452 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.670714  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.658885  8745 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:41.667671  8743 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.673949  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.667976  8824 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.675446  8889 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:41.668493  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.669191  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 121669us
I20260709 13:00:41.669571  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.676358  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.676869  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.678742  8820 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:41.684592  8745 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:41.690940  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.691591  8747 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:41.692437  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.693045  8747 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.689797  8823 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:41.656479 11437 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:41.596873 11055 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.702891  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.704329  8823 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:41.712853  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.607021 11449 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.695811 11437 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:41.853431  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.723757  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.881234  9447 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:41.722615  8893 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:41.812862  8824 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:41.733417  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.740556  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 619) took 1231 ms. Trace:
W20260709 13:00:41.850785  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 269035us
I20260709 13:00:41.903683  9442 rpcz_store.cc:276] 0709 13:00:40.509239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.384107 (+874868us) service_pool.cc:224] Handling call
0709 13:00:41.384413 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.384679 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.384708 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.385189 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.547971 (+162782us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:41.548005 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.548063 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.740368 (+192305us) raft_consensus.cc:1685] finished
0709 13:00:41.740386 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.740503 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129615,"threads_started":1,"wal-append.queue_time_us":250263}
W20260709 13:00:41.863044  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 181506us
I20260709 13:00:41.900118  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 611) took 1469 ms. Trace:
I20260709 13:00:41.735148  8746 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:41.905355  9440 rpcz_store.cc:276] 0709 13:00:40.430507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.684734 (+254227us) service_pool.cc:224] Handling call
0709 13:00:41.345952 (+661218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.401742 (+ 55790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.899934 (+498192us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400660dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:41.900060 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507866880}
I20260709 13:00:41.906280 11436 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:41.906378  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.907310  8746 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 4: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.908268  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.796442 11349 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:41.909759 11349 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.910041  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.910208 11349 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.910741  8816 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:41.713604  8745 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:41.908952  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:41.840622  8742 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:41.900792  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.916443  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.688239 11452 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:41.914074  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:41.921092  8743 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:41.922837  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:41.924425  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.925107  8748 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:00:41.918542  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.928359  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.929169  8742 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:00:41.929605  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 253429us
W20260709 13:00:41.741348 11456 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:00:41.937772  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.936884  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.938889  8746 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:41.932508  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 475561us
I20260709 13:00:41.845772 11466 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:41.941061 11466 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.941355  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.736219 10992 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.941969  8816 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.842825 11470 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:41.948808  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.842208 11055 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.927208  8749 raft_consensus.cc:2376] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6. Re-sending same reply.
I20260709 13:00:41.941452 11466 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.922009  8741 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:41.930135  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:41.948046  8863 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.721s (SENT)
I20260709 13:00:41.900828 11183 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:41.807585 11463 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:41.950487  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.967914  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.967900  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:41.934626  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 301531us
I20260709 13:00:41.826488 11326 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:00:41.843683 11270 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.905577 11407 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:41.954861  8742 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.955770  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:41.941991 11293 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:41.860631 11284 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:41.954074  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.959745  8816 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:41.996685  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:41.967147 11463 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:41.964414  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.814409 11469 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:41.969769  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:41.970469  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.801380 10909 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:00:41.734139 11464 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.245s
W20260709 13:00:41.977254  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.979312  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:41.980345  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.958387  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.963635 11409 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:00:41.933825  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:41.839242 11450 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:41.949229 11470 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:41.840531 11435 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:41.999461 11253 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.008756  8744 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:41.992856  8743 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:41.994665  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:41.994760  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:41.952606 11474 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.995821  9448 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:41.961334  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.011900  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:41.997581  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:41.997324  8821 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:00:41.996979 11473 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:00:42.016275 11470 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.017511 11470 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.017112  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 548) took 1225 ms. Trace:
I20260709 13:00:42.020839 11435 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.019969  9441 rpcz_store.cc:276] 0709 13:00:40.791878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.420434 (+628556us) service_pool.cc:224] Handling call
0709 13:00:41.420723 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.441693 (+ 20970us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:41.606311 (+164618us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.738115 (+131804us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b18003c8e00. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:41.918032 (+179917us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.943834 (+ 25802us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.943965 (+   131us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:41.943998 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.944059 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.016835 (+ 72776us) raft_consensus.cc:1685] finished
0709 13:00:42.016930 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.017054 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98872,"prepare.queue_time_us":154371,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":20609,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":10583,"spinlock_wait_cycles":9522432,"thread_start_us":353923,"threads_started":2,"wal-append.queue_time_us":234873}
I20260709 13:00:42.002614 11463 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.022907 11450 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.024170  8745 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:42.001857  8746 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:42.005340  8748 raft_consensus.cc:2376] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:00:42.025919  8745 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:41.982378  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.026481  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:42.026917  8742 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:42.006131  8823 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.005487  8749 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:42.010069  8741 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:41.999739  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.008262 11253 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.037060  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:42.046555  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.047307  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.066160  8748 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:42.080799  8746 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.089092  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.013638 11440 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.012884 11474 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:42.036049 11088 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:42.098342 11476 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:00:42.124575  8816 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:42.031190 11469 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.099718  8743 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:42.126860  8823 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:42.093820 11440 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.100212  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:42.106298  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 161951us
I20260709 13:00:42.090622  8822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:42.131914  8822 raft_consensus.cc:606] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:42.132094  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.112082 11450 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.114027 11289 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.114168 11451 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:42.115196 11471 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:00:42.145761  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 148318us
I20260709 13:00:42.065532 11435 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:42.120554 11467 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.259s	user 0.003s	sys 0.000s
I20260709 13:00:42.065647 11253 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.150585 11451 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.155352  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1354) took 1444 ms. Trace:
I20260709 13:00:42.122397 11449 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.155707  8815 rpcz_store.cc:276] 0709 13:00:40.710700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.710811 (+   111us) service_pool.cc:224] Handling call
0709 13:00:40.768085 (+ 57274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:40.773031 (+  4946us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:41.450945 (+677914us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18001008a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:00:41.526348 (+ 75403us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.713453 (+187105us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.716091 (+  2638us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.716280 (+   189us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:41.716312 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.716410 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.155033 (+438623us) raft_consensus.cc:1685] finished
0709 13:00:42.155125 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.155252 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14873,"prepare.queue_time_us":667627,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":5606,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":27778,"spinlock_wait_cycles":5433088,"thread_start_us":185830,"threads_started":2,"wal-append.queue_time_us":357815}
I20260709 13:00:41.987212 11437 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.065865 10992 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.157998 11437 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.158396 10992 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.158815  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.153847  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:42.121498 11281 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.589s	user 0.006s	sys 0.006s
I20260709 13:00:42.160573 10992 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.134828 11289 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.130383 11300 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.161047 11281 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.161602 11289 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.162313  8815 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:42.162663 11289 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.163544  8815 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.094523 11416 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.164067 11416 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:42.164618 11416 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.164842  8815 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:00:42.165611  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.166262  8815 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:42.167310  8815 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.093931 11326 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.138432 11469 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.109161 11474 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.139717 11450 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.171833  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.142426 11270 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.142920 11428 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.172448  8895 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.172526  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.173318 11281 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.173476  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.128225  8643 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:00:42.143968 11088 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:00:42.030802 11452 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.175452 11088 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.175956 11088 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:42.133615 11440 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.117508 11397 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.145125  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.178766  8743 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:42.147040  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:42.147408  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.179705  8893 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.149551 11468 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.150139 11475 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.154165  8822 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.154635  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:42.160472 11406 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.098889 11284 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.079823 11309 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:42.172001  8894 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:42.140724 11055 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.143425 11463 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.150496  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.204996  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.205544  8897 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.239807  8743 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.205900  8892 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.218698  8822 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:42.219978 11485 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:42.156821 11451 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.225701 11270 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:42.205157 11281 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:00:42.262481  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:42.177121 11088 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.251117  9440 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:42.253042  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 105738us
I20260709 13:00:42.264431  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.233525  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.260762 11253 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.255292 11449 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:00:42.267076  8896 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:42.240136 11309 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:42.282300  8893 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:42.252969 11406 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.289659  8815 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:42.206410 11466 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.293022  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.295915  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.299333  8743 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.275836 11270 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.308058  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.227073 11475 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.304004  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.288885 11406 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.306569  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.302618  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:42.335155  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.308146 11088 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.337308  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.341471 11011 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.342130 11468 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.342208 11011 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.343326  8897 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.344415 11253 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.338541  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.309106 11240 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.304296 11466 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:42.350976  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47092.9us
I20260709 13:00:42.302363 11490 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.345665  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.348153  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:42.351040  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48564.6us
I20260709 13:00:42.306569 11284 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.382421  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.382999  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:42.383270  8821 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.331270 11450 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.333226 11449 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.385563  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.353478 11490 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:42.384289 11289 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:42.384972  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.351039 11055 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.353402 11469 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.392220  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.390758 11490 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.388607 11300 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:42.334383 11281 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:42.399655  8746 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:42.400115  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48085.3us
I20260709 13:00:42.400825  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.396620  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:42.388761 10992 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.405514  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.406270 11446 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:42.396483 11055 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.399639 11300 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.396595 11397 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.411306 11296 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.349152 11240 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.420996 11281 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:42.421798 11463 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.422132 11240 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.422541 11281 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.423417 11240 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.417699 11397 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.424831 11397 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.405925  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.417116 11446 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.419433  8821 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:42.418853  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1366) took 1682 ms. Trace:
I20260709 13:00:42.430384 10909 raft_consensus.cc:2764] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:42.429821  8819 rpcz_store.cc:276] 0709 13:00:40.736047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.382006 (+645959us) service_pool.cc:224] Handling call
0709 13:00:41.382330 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.407913 (+ 25583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.418669 (+1010756us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:42.418807 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47614336}
I20260709 13:00:42.430215 11446 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.431007 10992 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.354148 11011 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.430349  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.417873 11300 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.438848 11436 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:42.413672 11480 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:00:42.391244 11309 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.435388 11011 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:42.441141 11436 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.411952 11270 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:42.455451  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 304920us
I20260709 13:00:42.457657  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:42.453294 11486 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:00:42.461176  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.437033 11349 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.458374 11011 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.465498 11449 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:00:42.478593 11055 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.457929 11436 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.462584 10992 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:42.481643 11475 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.482410 11440 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:42.511171 11487 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:00:42.450368 11416 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:42.523267  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.501267 11349 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.540145  8743 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.541071 11440 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.543385  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.544220  8743 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:42.545120 11427 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:42.470701 11402 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:00:42.526324 11449 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:42.552335  8749 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.527487 10992 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.545524 11427 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:42.557440  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 92982us
I20260709 13:00:42.560889  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.561328 10992 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.563146  8743 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.515829 11474 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.564222  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.566108  8748 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:42.566941  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.566870  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.568835  8746 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:42.570945  8747 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.551285 11449 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.573455  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.573792  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.574262  8740 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:42.574139  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.576117  8745 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:42.553828 11440 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:00:42.580994 11427 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.589994 11427 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.592072  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 625) took 1073 ms. Trace:
I20260709 13:00:42.592425  9449 rpcz_store.cc:276] 0709 13:00:41.518544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.591578 (+ 73034us) service_pool.cc:224] Handling call
0709 13:00:41.601635 (+ 10057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.798562 (+196927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.798606 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.798662 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:41.798709 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.591852 (+793143us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001955d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:42.591895 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.592007 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452524416}
I20260709 13:00:42.593596 11490 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.600837 11416 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.602846 11246 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.603300 11246 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.604075 11246 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.604903 11300 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.581892  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.586364 11449 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.587236 11474 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.584648 11397 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.611510  8743 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:42.613214  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:42.613778  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 1986) took 1086 ms. Trace:
I20260709 13:00:42.614526  8823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:42.615149  8742 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:42.618575  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 217884us
I20260709 13:00:42.653950  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:42.620286  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 159238us
I20260709 13:00:42.630837  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.632571  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.655252  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.655678  8820 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:42.612814  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1367) took 1876 ms. Trace:
I20260709 13:00:42.656744  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.628259  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.658448  8816 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:42.644065  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.646876  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.649686  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.649950  8889 rpcz_store.cc:276] 0709 13:00:41.527351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.539773 (+ 12422us) service_pool.cc:224] Handling call
0709 13:00:41.540024 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.675788 (+135764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.613550 (+937762us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:42.613720 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310878080}
W20260709 13:00:42.621227 11465 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.930s	user 0.000s	sys 0.012s
I20260709 13:00:42.665045  8819 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:42.624430 11460 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.937s	user 0.002s	sys 0.000s
I20260709 13:00:42.656395  8817 rpcz_store.cc:276] 0709 13:00:40.736297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.390429 (+654132us) service_pool.cc:224] Handling call
0709 13:00:41.390731 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.419600 (+ 28869us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:41.454424 (+ 34824us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.523629 (+ 69205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.523676 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:41.523718 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.612616 (+1088898us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b54002297d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:42.612658 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.612762 (+   104us) inbound_call.cc:177] Queueing success response
0709 13:00:42.623912 (+ 11150us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
Metrics: {"mutex_wait_us":34560,"prepare.queue_time_us":35278,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":625,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":17321,"spinlock_wait_cycles":115240576,"thread_start_us":68907,"threads_started":1,"wal-append.queue_time_us":182034}
I20260709 13:00:42.666572  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.665169  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.652478  8823 raft_consensus.cc:606] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:42.616045 11468 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:42.625420 10909 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:42.647975 11430 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.620450 11466 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.625836 11429 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.616135 11289 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.694877 11289 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.606077 11290 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.614125 11478 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:42.675120  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.621260 11490 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:42.677570 11240 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.614032 11011 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.605265 11246 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.643505 11300 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:42.684197  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:42.691704 11483 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.672s	user 0.002s	sys 0.000s
I20260709 13:00:42.693922 11437 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:42.668128 11296 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.669373 11440 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:42.647738 11426 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:00:42.619839 11469 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.613763 11436 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.727996 11436 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:42.729414  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.680796  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.679030 11326 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.683172 11474 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.685123 11449 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.730821  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7884996e8b84433f80124e36111f58ac"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:42.732084 11397 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.737584  8643 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 5, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 5 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:42.739574  8643 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index -1 to 4, term changed from 4 to 5, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to <none>, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:00:42.702157 11429 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.743414 11429 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:42.744328  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.749784  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.751920 11469 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:42.750955  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.752663 11300 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:42.752674 11421 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.715441 11296 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.753266 11469 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:42.753619 11296 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.753561 11421 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.753268 11300 consensus_queue.cc:1243] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:42.754753  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.713430 11407 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:42.757097 11407 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.757882 11407 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.758975 11407 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.710410  8817 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:42.761344  8819 raft_consensus.cc:606] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.761710  8819 raft_consensus.cc:614] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:42.762953 11421 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.715617 11437 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:42.764061  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.764703  8819 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.764532 11437 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.765794 11437 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.766734  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 1991) took 1270 ms. Trace:
I20260709 13:00:42.767098  8824 rpcz_store.cc:276] 0709 13:00:41.496785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.519283 (+ 22498us) service_pool.cc:224] Handling call
0709 13:00:42.766529 (+1247246us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400076a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:42.766646 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256460032}
I20260709 13:00:42.770012 11296 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.718463  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:42.721060  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 64425.5us
I20260709 13:00:42.701769 11468 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.677925 11406 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.788578  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.723404 11490 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.724535 11011 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.685491 10909 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.789389 11468 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.791393 10909 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.702783 11478 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:42.791756 11406 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.684911 11281 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:00:42.704398 11290 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.737600 11474 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:42.770679  8815 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:42.772260 11427 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.716957  8823 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.815963  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.774520 11463 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.714566  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:42.779048 11507 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:00:42.830618 11296 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.779392 11417 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:42.831306 11296 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.831687 11417 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.832144 11296 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.832535 11296 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:42.779826 11253 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.828105 11427 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.833276 11296 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.853366  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:42.853672  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 149935us
I20260709 13:00:42.781685 11390 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:42.782308 11246 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:42.783767 11397 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:42.784827  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.786432  8742 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:42.880589  8745 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.697932 11430 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.792676  8821 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.792874  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.795109  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.834062 11253 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.802381  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.885083  8819 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.804584  8822 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:42.814148  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 94020.2us
W20260709 13:00:42.889616  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.827435  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.827832  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.818070  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.826139  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:42.829304 11428 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:42.829389  9445 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.829901 11421 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.830246 11281 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.780929 11485 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.816675 11440 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:00:42.833834 11417 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.836930  8816 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:42.855352  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:42.854030  8815 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:42.915022  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.915731  8815 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:42.845131 11406 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:00:42.882582  8743 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:42.916719  8745 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:42.795796  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:00:42.882584  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 89846us
I20260709 13:00:42.892524  8749 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.827119 11463 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:42.839674 11290 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.893945  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.899734  8823 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.922331  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.922353  8821 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.923532  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.839921 11427 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.842653  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.924192  8815 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:42.912328  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 94382.1us
I20260709 13:00:42.924484  8817 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.924979  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.887523 11449 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:42.810060 11478 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.865881  8818 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:42.918924  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 164338us
I20260709 13:00:42.893129 11428 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:42.883111 11055 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:00:42.944831  8821 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:00:42.922307  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 241690us
I20260709 13:00:42.945961  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.909883  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:42.901681 11296 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.947435  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:42.916723  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:42.926069  8712 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.513s (SENT) [suppressed 42 similar messages]
I20260709 13:00:42.933179  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.935755  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.949957  8889 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:42.950349  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.950464  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.951536  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:42.952366  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.939544  8891 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:42.913607 11183 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.954767  8898 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:42.954480  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:42.942044  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 146407us
I20260709 13:00:42.911715 11421 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.965224  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.966282  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.966976  8816 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:42.967634  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.968438  8816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:42.968911  8816 raft_consensus.cc:606] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:42.913558 11485 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.969863  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:42.970958  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:42.971800  8821 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:42.972911  8821 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:42.973522  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.973784 11270 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.974274  8890 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:42.975358  8890 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:00:42.976089 11515 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:42.976483  8890 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:42.978174 11430 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:42.956172  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:42.978897  8892 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.980029 11463 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:42.981026 11463 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.981215 11270 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.928992 11440 consensus_queue.cc:1243] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:00:42.958246  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.982183  8889 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:42.958875  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:42.982900  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:42.932060 11490 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:00:42.985952 11349 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:42.986536 11349 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:42.987382 11349 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:42.961536  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:42.954442 11240 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:00:42.990710  8815 pending_rounds.cc:85] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 5
I20260709 13:00:42.990969  8815 pending_rounds.cc:107] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:42.991210  8815 raft_consensus.cc:2889] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:42.992437 11253 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:42.954674 11495 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.993885 11495 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.994230 11495 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.954559 11469 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.997628 11469 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:42.997967 11469 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.999425  8643 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.958153 11326 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.001686 11326 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.002063 11326 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.002126  8647 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 283) took 21307 ms (client timeout 15000 ms). Trace:
I20260709 13:00:43.006006 11520 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:43.006263  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 73208.5us
I20260709 13:00:43.006795 11514 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.007186 11514 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:43.007532 11514 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.957903  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:43.008335  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 56068.6us
I20260709 13:00:43.009274  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:43.010228 11290 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:43.009506  8643 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 3, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.011837 11055 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.012214 11055 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.012542 11055 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.013454 11440 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:43.013841 11440 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:43.014626 11515 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:43.012744  8643 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 3, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.014938 11426 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:43.015358 11426 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:43.015538 11515 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.016249 11426 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.017470 11426 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.969192  8816 raft_consensus.cc:614] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:43.007391  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.021554  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.023965  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:43.024729 11281 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:43.025389 11390 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:43.025859 11281 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.026232 11281 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.026549 11281 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.929661 11300 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.037159  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.029641  8896 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:42.958890  8893 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:43.031867  8892 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:42.957033  8823 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:43.040045  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.046485  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.048069  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:42.942754 11427 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.078166  8898 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.059693  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.080713  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:43.060216 11478 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.060303  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
W20260709 13:00:43.061111 11518 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:00:43.061530 11512 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:00:43.002560  8647 rpcz_store.cc:269] 0709 13:00:21.694762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.715355 (+ 20593us) service_pool.cc:224] Handling call
0709 13:00:26.436178 (+4720823us) spinlock_profiling.cc:243] Waited 4.37 s on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:00:43.001892 (+16565714us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:32.522921 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:32.612005 (+ 89084us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:32.612063 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:00:32.612100 (+    37us) tablet.cc:662] Decoding operations
0709 13:00:32.842713 (+230613us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:32.842814 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:32.842853 (+    39us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:00:32.845119 (+  2266us) tablet.cc:701] Row locks acquired
0709 13:00:32.845172 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:00:33.003890 (+158718us) write_op.cc:270] Start()
0709 13:00:33.036843 (+ 32953us) write_op.cc:276] Timestamp: P: 1783602032877951 usec, L: 0
0709 13:00:33.041082 (+  4239us) op_driver.cc:348] REPLICATION: starting
0709 13:00:33.233690 (+192608us) log.cc:844] Serialized 8693 byte log entry
0709 13:00:34.077533 (+843843us) op_driver.cc:464] REPLICATION: finished
0709 13:00:34.600914 (+523381us) write_op.cc:301] APPLY: starting
0709 13:00:34.613878 (+ 12964us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:34.632989 (+ 19111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:34.633124 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:34.757572 (+124448us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:34.757983 (+   411us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:00:34.758027 (+    44us) write_op.cc:312] APPLY: finished
0709 13:00:34.758508 (+   481us) log.cc:844] Serialized 152 byte log entry
0709 13:00:35.185061 (+426553us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:35.206718 (+ 21657us) write_op.cc:454] Released schema lock
0709 13:00:35.313877 (+107159us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:35.314267 (+   390us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:35.314295 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":61303,"reported_tablets":40,"spinlock_wait_cycles":10052220672,"child_traces":[["op",{"apply.queue_time_us":479902,"apply.run_cpu_time_us":17701,"apply.run_wall_time_us":729787,"num_ops":22,"prepare.queue_time_us":348417,"prepare.run_cpu_time_us":30308,"prepare.run_wall_time_us":802555,"raft.queue_time_us":729817,"raft.run_cpu_time_us":63666,"raft.run_wall_time_us":339171,"replication_time_us":1036290,"thread_start_us":1014628,"threads_started":5,"wal-append.queue_time_us":635642,"wal-append.run_cpu_time_us":32015,"wal-append.run_wall_time_us":2268590}]]}
I20260709 13:00:42.952054 11417 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:43.081924  8819 raft_consensus.cc:606] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.104086  8819 raft_consensus.cc:614] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:43.093163  8817 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:42.948793 11449 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:43.095487  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:43.100194  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.006562 11520 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:43.120224  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.118216  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.108215  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:43.103642 11426 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:43.151805  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 225578us
I20260709 13:00:43.130116  8897 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:43.038830 11490 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:00:43.159133  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.081907 11390 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:00:43.184707  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.094928 11416 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:00:43.188984  8895 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.101735 11441 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:00:43.192339  8897 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.101670 11427 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.152477 11290 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.129158 11480 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:43.197196  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 155941us
I20260709 13:00:43.198423 11480 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:43.105878 11519 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:43.213119  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 203994us
I20260709 13:00:43.214298  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.214783  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:43.108541 11326 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:43.223266  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.220932  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:43.230677 11427 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.108225 11428 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.233650  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.233777  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.197443 11290 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.237253  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:43.258241  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.210894 11519 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:43.259572  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.260028  8896 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:43.196735 11426 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:00:43.266780  8895 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:43.267290  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.270231  8897 raft_consensus.cc:2368] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7.
I20260709 13:00:43.221841 11430 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:43.273313  8893 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.200407 11480 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.257682 11290 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.213117 11469 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.292316  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.266716 11519 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.304765  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.262156 11427 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:00:43.263686 11281 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.238747 11449 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.287352 11528 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.269534 11514 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.252691 11520 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.317410 11427 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:43.331344  8891 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:43.317633 11421 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.335583 11534 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:43.308800 11055 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:43.332293  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 173359us
I20260709 13:00:43.342854  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.347573  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:43.354262  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:43.333142 11427 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:43.367321 11527 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:00:43.369503 11524 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:00:43.371320 11530 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:00:43.373523  8895 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.375322  8889 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:43.371495  8891 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.379868  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.377000  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.385582  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.386387  8894 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:43.391963  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.323545 11528 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.406687  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:43.410156 11537 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.410763 11537 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.411223 11537 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:43.331466  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 277792us
I20260709 13:00:43.420469  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:43.429270  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:43.430141  8822 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:43.413439 11417 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:43.432282  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.434479  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.435053  9442 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:43.435981  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:43.437731  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:43.438314  9440 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:43.415699 11326 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:43.421044 11485 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:43.443120  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:43.444118  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.444202  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.446367  8822 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:43.446403  9442 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.444085 11526 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:00:43.448138  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:43.448756  9445 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:43.453183  9442 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.448143 11417 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.454898  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.455721  9442 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:43.448757  8821 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:43.460583 11495 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.479861  9440 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:43.501519  9440 pending_rounds.cc:85] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 5
I20260709 13:00:43.501982  9440 pending_rounds.cc:107] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:00:43.502281  9440 raft_consensus.cc:2889] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.502988  8893 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.506253  8749 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.444622 11469 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:43.515581  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.428231 11427 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.520449  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:43.520558  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.521911  8747 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.525458  9441 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:43.517452  8745 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.525698  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:43.526962  8744 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:43.521659  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.531411  8745 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.507618  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.536666  8742 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:43.532953  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.533658  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:43.532845  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 386420us
W20260709 13:00:43.536705  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.537611  8749 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.537989  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.538543  8741 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:43.534974  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:43.540575  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 134055us
I20260709 13:00:43.541750  8747 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.542508  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:43.511612 11281 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:43.543777  9441 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.546350  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.547343  8744 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:43.549713  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.548925  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:43.524278 11416 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:43.561889  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.513265 11309 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:43.518647 11541 raft_consensus.cc:2764] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:43.525336 11441 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.578464  8744 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:43.548539 11390 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.553534 11427 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.594849 11540 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:00:43.595162  8745 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:43.595026 11390 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.616065  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.595712 11441 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:43.596879  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.584926  9445 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:43.603039 11428 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.607407 11539 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:00:43.608547  8749 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:43.557749 11417 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.553872 11520 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.591764  9444 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:43.566262 11055 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.648061  9442 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:00:43.619169 11478 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
I20260709 13:00:43.574841 11281 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.585711 11309 consensus_queue.cc:1243] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:43.592096 11416 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.591163 11544 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.644127  8746 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:43.648722 11520 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.625458 11541 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:43.602766  8741 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:00:43.671257  8748 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:43.689128  8746 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:43.694487  9449 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:00:43.694823  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.700889 11240 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:43.701476 11240 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:43.702522 11240 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.703368  8821 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:43.703780 11240 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.707279  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.707815  8822 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.710896  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.712615  8823 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:43.647440 11519 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.647002 11417 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:43.721438  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.720983  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:43.727185  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:43.716467  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.740082  8889 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:43.729450  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.741161  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:43.710706 11416 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:43.765658  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.767279  8889 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.710203 11544 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:43.765417  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.767632  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.767446  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:43.715440 11495 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:43.686403 11281 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:43.764964  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.774529  8745 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.795943  8745 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.797060  8822 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:43.730993 11538 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.768786 11466 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:43.806766  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.714689 11541 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:43.821275 11548 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:00:43.822275 11556 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:00:43.787077 11536 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:00:43.785717 11469 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:43.830634 11552 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:00:43.812100  9445 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:00:43.840494 11545 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:00:43.840348 11544 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.838591 11485 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:43.840996  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.829583 11538 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:43.858191 11538 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:43.869591  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.871948  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:43.874284  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.875501  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:43.875859  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.876119  8894 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:43.877013  8820 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:43.829069 11466 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:43.877903  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.878801  8896 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:43.885380  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:43.885735  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.887002  8745 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:43.892048  8745 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:43.899803  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.900566  8747 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.906152  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:43.909237  9440 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:00:43.913990  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.919706  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:43.920485  8746 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:43.905530 11557 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:00:43.881433 11011 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.846428 11300 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:00:43.922807  8745 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.882052 11532 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:43.899255 11469 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:43.933521  8889 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:43.946053  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:43.946852  9440 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:43.946957  8745 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.964562 11549 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:00:43.968664 11559 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:00:43.970988 11520 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:43.976267 11537 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.003s
I20260709 13:00:43.980578 11528 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.987875 11281 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.873852 11416 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.943867 11519 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.004562  8746 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:44.015372  8748 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.938254 11011 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:43.976876 11537 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.027297  8745 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:44.005380 11417 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:43.963394 11449 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:00:44.050990  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:43.945048 11532 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.075696  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:44.024744 11463 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:44.079586  8745 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:44.080507  8740 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:44.088245  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:44.089720  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:44.091194  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.091908  9449 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:44.092648  8894 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:44.089337  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:44.101369 11563 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:00:44.105114 11537 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.105779  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:44.107649 11532 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.058785 11469 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.109927  8895 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:00:44.112888 11564 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:00:44.061151  9445 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:44.119872 11390 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.125656 11440 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.126875 11440 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.129516 11270 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.130079 11270 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.133893  8819 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:00:44.078038 11449 consensus_queue.cc:1243] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:44.079869 11463 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:44.083957 11562 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:00:44.083084  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.019879 11466 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:44.140902  8893 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:44.266009  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.150774  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:44.286988  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.308915  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.266119  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.312582  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:44.315361  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:44.146302  8740 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:44.320425  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:44.322688  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:44.315652  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:44.310667  8744 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:44.316191  8889 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:44.318301  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:44.301457 11290 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.000s	user 0.005s	sys 0.009s
W20260709 13:00:44.263445 11555 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.427s	user 0.004s	sys 0.001s
W20260709 13:00:44.260130 11565 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:44.327370  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:44.318013  9449 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.327921  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:44.342056  8821 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:44.342193  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.342782  8892 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:44.337723  8894 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:44.282351 11421 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:44.343709  8897 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:00:44.321530  8747 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:44.344841  8745 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.317099  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:44.270038 11535 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:00:44.336621  8898 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:44.336974  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:44.257097  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.319645 11490 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:00:44.344671 11547 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:00:44.343112 11577 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:44.305472 11428 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.656s	user 0.008s	sys 0.004s
I20260709 13:00:44.353710 11240 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.354463 11541 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:44.315557 11011 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.315102 11270 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.358552  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:44.358973 11427 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:44.360512  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43507.6us
I20260709 13:00:44.361202  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:44.334724  9412 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.244469 11573 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.391124 11573 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.320489 11560 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:44.404901  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.360185 11421 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:44.375870 11463 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.376744 11490 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.328398 11576 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:44.383174  8891 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:44.389995  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55416.4us
I20260709 13:00:44.567843 11490 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:44.568151  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:44.327628 11409 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:00:44.385088 11427 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:44.569129 11490 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:44.569697 11427 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.377365  9444 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:44.412833 11560 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.572479 11560 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:44.344324 11290 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.414160 11573 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:44.420382  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.578954  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.377489 11577 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.581075 11577 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.412597 11474 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:44.488464  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.581643  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 176916us
I20260709 13:00:44.582069 11474 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.582705 11474 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:44.582998 11474 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:44.583775 11474 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.584086 11240 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:44.585383 11183 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:44.586912 11349 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:44.587807  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.588745  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.588871 11349 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:44.589584 11469 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.590293  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.591162 11468 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:44.591432 11469 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:44.592132 11469 consensus_queue.cc:1243] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:44.584920 11474 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:44.404170 11428 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.371616  9445 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:44.593472 11485 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:44.500914  9446 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:44.567260 11576 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.570802 11463 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.594312 11576 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.596036 11349 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:44.573240  9441 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:44.597081 11349 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.597813 11011 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.598270 11011 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.491043 11421 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.591718 11468 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:44.600240 11011 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.577569 11290 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.601356 11468 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.602500 11576 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.604586 11576 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.594095 11485 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.607069 11519 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.607250 11485 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:44.607640 11519 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.609228 11573 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.611588  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.616372  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:44.620118  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.627745  9442 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:44.631268  9442 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.631356  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.617713 11440 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.630429 11490 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.629683 11290 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:44.640576  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.014s	sys 0.023s
W20260709 13:00:44.642416  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.015s	sys 0.023s
W20260709 13:00:44.642838  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 765724us
I20260709 13:00:44.643527  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.630589 11532 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:44.673890  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51105.8us
W20260709 13:00:44.679188  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.631098 11569 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:00:44.686923  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.709645 11011 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:00:44.714447  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 577) took 1099 ms. Trace:
I20260709 13:00:44.684201 11532 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:44.714898  9441 rpcz_store.cc:276] 0709 13:00:43.615475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.615762 (+   287us) service_pool.cc:224] Handling call
0709 13:00:44.714250 (+1098488us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b540065e854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:44.714367 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749225216}
I20260709 13:00:44.715310 11011 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:44.715709 11532 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.716444  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 572) took 1211 ms. Trace:
I20260709 13:00:44.716688 11592 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.716820  9446 rpcz_store.cc:276] 0709 13:00:43.505418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.505552 (+   134us) service_pool.cc:224] Handling call
0709 13:00:44.716294 (+1210742us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540059f6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:44.716391 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1421662976}
I20260709 13:00:44.666926 11290 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.717510 11592 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:44.718744 11592 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.719504 11290 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.720302 11592 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.730436 11290 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.731177 11290 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:44.732687  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.714778 11569 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:44.740940 11569 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.741315 11569 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:44.742760 11558 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.918s	user 0.003s	sys 0.000s
I20260709 13:00:44.720324  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.751204 11427 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.758914 11592 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.759590 11592 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.760649 11569 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:44.770073  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 95337.9us
I20260709 13:00:44.771524  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.819051 11468 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.824038 11468 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.826579 11468 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.822131 11547 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:44.828219 11409 consensus_queue.cc:1243] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:00:44.828912 11409 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.829370 11409 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:44.829742 11290 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.830547 11409 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.832186 11409 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:44.833649  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 113450us
I20260709 13:00:44.834698  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.826627 11592 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:44.837699 11541 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:44.837874 11535 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.838246 11541 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.838436 11535 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.838833 11541 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.839367 11569 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:44.839553 11416 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:44.838909 11535 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:44.840860  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 197513us
W20260709 13:00:44.842270  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 499622us
I20260709 13:00:44.842680  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.843608  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:44.860574  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71357.4us
I20260709 13:00:44.845644 11592 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.821357 11390 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:00:44.856184 11569 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:44.858501 11597 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:44.875245  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 103859us
I20260709 13:00:44.883620  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:44.869115 11474 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:44.891898  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1418) took 1042 ms. Trace:
I20260709 13:00:44.892227  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:44.878542 11597 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:44.884567 11326 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.892424  8743 rpcz_store.cc:276] 0709 13:00:43.849983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.850144 (+   161us) service_pool.cc:224] Handling call
0709 13:00:43.850507 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.850733 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:44.001754 (+151021us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.341726 (+339972us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.359703 (+ 17977us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800380da0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:44.772721 (+413018us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b1800380da0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:00:44.891565 (+118844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:44.891616 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:44.891661 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.891724 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.891840 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":485739,"prepare.queue_time_us":37330,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":74,"raft.queue_time_us":441030,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":39875,"spinlock_wait_cycles":47712896,"thread_start_us":571423,"threads_started":3,"wal-append.queue_time_us":375983}
I20260709 13:00:44.879307 11592 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.890457 11598 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.880435 11569 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.896082 11595 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.891274 11474 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.896274 11416 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:00:44.903378 11326 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.925873 11468 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.926443 11468 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:44.928731 11468 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.923914  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:44.946985  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 88301.2us
I20260709 13:00:44.950452  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:44.952050  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:44.916458 11390 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:00:44.923594 11596 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:44.922978 11602 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:44.958865  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.932713 11597 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.960572 11390 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:44.960796 11597 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.961115 11390 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:44.962029 11390 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.964531 11490 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.915946 11603 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:00:44.968819 11603 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.969197 11603 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:44.977547  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 140372us
I20260709 13:00:44.980791  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:44.990444  8747 pending_rounds.cc:85] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 2
I20260709 13:00:44.990759  8747 pending_rounds.cc:107] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:00:44.991510  8747 raft_consensus.cc:2889] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:45.000034  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 156575us
I20260709 13:00:45.000726  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:45.002264  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65776.8us
I20260709 13:00:45.002672  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:45.003360  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.004832  8749 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:45.001370  8744 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:45.010003 11290 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:45.010591 11290 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.013926 11290 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:45.014727 11519 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.015944  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:45.018929  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:45.016896 11514 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:44.917088 11595 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.965065 11490 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:44.991312 11485 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.044049 11592 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.044260 11326 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.044715 11592 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.045359 11592 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:45.045564  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.045879 11592 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.046298  8742 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:45.047468  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:44.990528 11608 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:45.002468 11441 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:45.054809  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.035193 11569 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.070463  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.073397  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:45.033102 11603 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.074293  8740 raft_consensus.cc:2376] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:00:44.991091 11604 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:00:45.048902 11253 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.084741 11611 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:45.085964 11569 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:00:45.090288 11566 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.985s	user 0.007s	sys 0.007s
I20260709 13:00:45.090927 11605 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.091521 11605 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:45.093395  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.048173 11612 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:45.098636 11612 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:45.099715 11612 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.072199 11441 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.109087 11441 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.109398 11253 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.111522 11253 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.111589 11535 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.050661 11485 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.065225 11532 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:45.118750  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.119200 11532 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.076773 11608 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:45.121559 11608 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.125676 11604 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.126101 11604 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:00:45.129176  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.130432  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 129507us
I20260709 13:00:45.135964  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:45.169768  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.181838  8824 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:00:45.182319  8824 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:45.182565  8824 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:00:45.182834  8824 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:45.177425 11609 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:00:45.118806 11474 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.177640 11519 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:45.179952 11603 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.177320 11613 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:45.168797 11596 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:45.177531 11514 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:00:45.177296 11592 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:00:45.246001  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.246191  8740 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:45.226442 11595 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.238600 11613 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:45.239660 11541 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.236768 11519 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.217088 11604 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:45.614635  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.269356  8816 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:00:45.262486  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.264972  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.269182  8815 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:45.398945  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.383414 11532 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.623955  8745 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:00:45.458393  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.333920 11605 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.624426  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:45.623497  9449 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:45.343438  9445 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:00:45.628497  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.631421  8744 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:45.621141  8824 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:00:45.635036  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.636236  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.262229 11613 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.265333  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.292524  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.636538  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.628203 11519 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:45.392432  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.311237  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 270734us
I20260709 13:00:45.637669  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.655340  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.523877 11596 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:45.696218  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 466844us
I20260709 13:00:45.700605  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:45.701421  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.259775 11485 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.616989 11608 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.377723 11474 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:45.618467 11625 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:00:45.343873 11490 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.252337 11602 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:45.443077 11427 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.001s
I20260709 13:00:45.636795  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:45.500109 11598 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.000s
W20260709 13:00:45.788650  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 520853us
W20260709 13:00:45.820647  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.800122  9446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.023s	sys 0.030s
W20260709 13:00:45.821585  9446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.023s	sys 0.030s
W20260709 13:00:45.801365  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.822382  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.801241  8816 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:45.295251 11595 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.514s
I20260709 13:00:45.451973 11617 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.801975  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:45.261719 11535 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.828979  8896 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:45.382861 11624 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:45.803884 11519 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.806231  8787 leader_election.cc:400] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7.
I20260709 13:00:45.835378  8787 leader_election.cc:403] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:00:45.381661 11619 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.516403 11616 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:45.574278 11606 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.813618  8821 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:00:45.821736  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:45.827687 11427 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:45.725965  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:45.838243 11427 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.838507 11616 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.838096  8890 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:45.723819 11390 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 13:00:45.727546  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:45.852265 11616 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.846455 11619 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:45.862617  8819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 429) took 1987 ms (client timeout 1764 ms). Trace:
I20260709 13:00:45.863044  8818 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:45.864112 11619 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.836417 11624 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:45.865211 11623 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:00:45.866683 11624 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:45.867587 11626 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:00:45.867782  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67899.8us
I20260709 13:00:45.871240 11546 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.871704 11546 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:45.869019 11449 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7.
I20260709 13:00:45.825604 11598 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.841024 11535 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:00:45.873229  8787 connection.cc:256] RPC call timeout handler was delayed by -1.0267565080000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:00:45.878573  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:45.888355  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:45.876528 11595 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:45.889515 11595 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.890015 11595 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.879081 11606 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.891142  8740 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:00:45.833403  8746 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.003s	sys 0.005s
W20260709 13:00:45.834264  8748 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.008s	sys 0.008s
W20260709 13:00:45.863058  8819 rpcz_store.cc:269] 0709 13:00:43.875491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.875641 (+   150us) service_pool.cc:224] Handling call
0709 13:00:45.862430 (+1986789us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:45.862553 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591696640}
I20260709 13:00:45.814499 11514 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.813974 11532 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.703446 11621 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.169s
W20260709 13:00:45.901902  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.904100  8712 connection.cc:256] RPC call timeout handler was delayed by -1.9499488179999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:00:45.904803  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.824976  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 186377us
W20260709 13:00:45.893724  8746 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.004s	sys 0.005s
W20260709 13:00:45.894171  8748 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.008s
W20260709 13:00:45.883298 11597 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.922s	user 0.009s	sys 0.000s
I20260709 13:00:45.817538 11474 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:45.913471  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.817327 11608 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:45.876223 11629 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.876363 11620 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:45.908381 11551 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 2.138s	user 0.016s	sys 0.009s
I20260709 13:00:45.897636 11535 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:45.933537 11547 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:45.938097 11606 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.834966 11617 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.943336 11617 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.948706 11426 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.949443 11426 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.949822 11608 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.949913 11426 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.951217  8816 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:00:45.951568  8816 pending_rounds.cc:85] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 7
I20260709 13:00:45.951907  8816 pending_rounds.cc:107] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:00:45.951440 11595 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.952126  8816 raft_consensus.cc:2889] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:45.952558 11426 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.954056 11595 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:45.916211  8712 connection.cc:256] RPC call timeout handler was delayed by -1.4940529799999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 13:00:45.954638  8712 connection.cc:256] RPC call timeout handler was delayed by -1.4481497069999705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23210000000000003s for a response
W20260709 13:00:45.955116  8712 connection.cc:256] RPC call timeout handler was delayed by -1.0002335849999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2271s for a response
I20260709 13:00:45.955189 11541 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:45.873464 11602 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.956413  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.957696  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:45.959364  8748 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:45.910095 11417 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.900522 11290 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.875049 11449 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:00:45.841274 11519 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.938809 11547 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.953069 11426 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:00:45.938935 11597 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.909719 11514 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.939497 11474 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.975955 11597 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.977497 11547 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:45.986202  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.982308  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:45.984377  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:45.988183  8824 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:45.997219  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:45.912703 11532 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.012260  8824 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:46.024187  8787 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.537s (SENT)
I20260709 13:00:46.024811  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.026288 11618 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:45.961177 11629 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:45.961068 11620 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.996999 11595 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.034076  8747 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:00:46.037418  8747 pending_rounds.cc:85] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 6
I20260709 13:00:46.037678  8747 pending_rounds.cc:107] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:00:46.037935  8747 raft_consensus.cc:2889] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.038656  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:46.039386  8742 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:46.039502 11593 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.201s	user 0.003s	sys 0.021s
I20260709 13:00:46.040335 11613 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:45.966482 11417 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:46.045936  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.054920  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2013) took 2609 ms. Trace:
I20260709 13:00:46.055330  8817 rpcz_store.cc:276] 0709 13:00:43.445923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.446052 (+   129us) service_pool.cc:224] Handling call
0709 13:00:43.501504 (+ 55452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.501975 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.502024 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.517960 (+ 15936us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.647236 (+129276us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:43.647277 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.647977 (+   700us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.262801 (+614824us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:46.053595 (+1790794us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b24000e9948. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f3420fb0 00007f90f6097488 00007f90f6097be8 00007f90f606f398 00007f90f606a76a 00007f90f6069bb6
0709 13:00:46.054646 (+  1051us) raft_consensus.cc:1685] finished
0709 13:00:46.054675 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.054860 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":896,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5715,"spinlock_wait_cycles":24880128,"thread_start_us":81867,"threads_started":1,"wal-append.queue_time_us":220204}
I20260709 13:00:46.071010  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.076426  8740 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:00:46.036368  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 310618us
W20260709 13:00:46.077868  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 189727us
W20260709 13:00:46.084952  8862 connection.cc:256] RPC call timeout handler was delayed by -1.7519030180000073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 13:00:46.086162  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:46.086637  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.088413  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:46.091053  8862 connection.cc:256] RPC call timeout handler was delayed by -1.404503018000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:00:46.091617  8862 connection.cc:256] RPC call timeout handler was delayed by -1.2479030179999882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 13:00:46.092114  8862 connection.cc:256] RPC call timeout handler was delayed by -1.2344030180000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 13:00:46.092525  8862 connection.cc:256] RPC call timeout handler was delayed by -1.1885030180000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
I20260709 13:00:46.093966  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:46.095019  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:46.075337 11544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.972s	user 0.023s	sys 0.017s
I20260709 13:00:46.099229  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.094923  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:46.103171  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.105242  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:46.106385  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 203007us
I20260709 13:00:46.104955  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.048183 11595 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.108404  8896 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:46.090582 11270 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.111613  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.094513 11474 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.117633  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:46.106828 11603 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.863s	user 0.009s	sys 0.002s
I20260709 13:00:46.093904  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 636) took 1462 ms. Trace:
I20260709 13:00:46.112260  8890 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
W20260709 13:00:46.099423 11544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.996s	user 0.023s	sys 0.017s
I20260709 13:00:46.113329  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.114050  8898 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:00:46.124317  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:46.124879  8897 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.104271 11644 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.117662  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.110333  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.110903  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:46.118896  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.104099 11326 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.114745  8896 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.112696 11621 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:46.126255  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.098299 11290 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:00:46.132061  8892 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:46.133785  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.141218  8820 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:46.126533 11597 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.147266  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.118824 11646 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.111516 11011 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:00:46.144534 11617 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.121030 11602 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:00:46.146693 11644 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.215834  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.148309 11326 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.121332 11270 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.203965  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:46.205093  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 71394us
I20260709 13:00:46.123368 11474 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.100067 11649 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:00:46.206940  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.144135 11603 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.208217  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.155809 11651 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.213071  8897 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:46.212924  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 95391.1us
W20260709 13:00:46.213675  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 118873us
I20260709 13:00:46.265110  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:46.266556  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:46.266884  8742 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:00:46.266947  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.240407  8892 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.146018 11648 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:00:46.260206  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.209059 11597 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:46.261178  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.262168  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.272197  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.266911  8898 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:46.244215  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.269877  8890 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:46.270968  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.273569  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.273978  8897 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:46.271255  8895 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:46.271651  8891 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:46.274703  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.276151 11621 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.276564  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.277302  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.277689  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:46.278254 11654 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:00:46.274729  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.278430 11576 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:46.278777  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.278891  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:46.265302  8894 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:46.280061 11576 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.281350 11485 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:46.281368 11576 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:46.279906  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.279963  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:46.278425  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:46.280853  8893 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:46.259140 11542 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:46.284439  8898 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:46.287384  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.284421  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:46.288949  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:46.289644  8820 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:46.289979  8820 pending_rounds.cc:85] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 3
I20260709 13:00:46.290282  8820 pending_rounds.cc:107] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:46.290521  8820 raft_consensus.cc:2889] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.292187  8823 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:46.292413  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.286252  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.293824  8823 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:46.297560 11592 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.280282 11490 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:46.298028 11547 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:46.298349 11490 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:46.298457  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:46.298789 11490 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.298919 11547 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:46.300335  8890 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:46.285306 11576 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.298821  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.301810  8742 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:46.268324 11602 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.308182  8823 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.304620  8820 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.308339  8897 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.312582  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.265666 11651 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:46.286810 11532 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.318131 11541 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.283252 11613 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.287526 11542 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.299983 11547 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:00:46.315759  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:46.301679 11618 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.344630  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.332960  8823 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:46.348356  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:46.346124 11542 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:46.124042  9444 rpcz_store.cc:276] 0709 13:00:44.626260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.629684 (+  3424us) service_pool.cc:224] Handling call
0709 13:00:44.654215 (+ 24531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.654711 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:44.655088 (+   377us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.771641 (+116553us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:44.818795 (+ 47154us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.842200 (+ 23405us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:44.846024 (+  3824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.858235 (+ 12211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:46.038983 (+1180748us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:46.045714 (+  6731us) raft_consensus.cc:1685] finished
0709 13:00:46.072324 (+ 26610us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.088561 (+ 16237us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":306,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":20685,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":19374,"spinlock_wait_cycles":6333184,"thread_start_us":85103,"threads_started":1,"wal-append.queue_time_us":183120}
W20260709 13:00:46.351992 11650 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:46.345008 11409 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:46.314640 11349 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.353585  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.307186 11417 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.304364 11658 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.338419 11485 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:46.298101 11592 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:46.341100  8820 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:46.302884 11599 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:46.239842 11646 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.316202 11534 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:46.320349 11661 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.071s	user 0.003s	sys 0.010s
W20260709 13:00:46.333359  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66895us
I20260709 13:00:46.315142 11519 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.270753 11603 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:46.365787  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 443) took 1099 ms. Trace:
I20260709 13:00:46.323348 11651 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:46.390340 11645 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:00:46.394012  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.374377 11592 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.398299 11270 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.399638  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.400300  8820 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:46.384263 11290 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:00:46.406368  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.346004 11532 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.406983 11603 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.407933  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.331828 11656 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:46.409018 11656 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:46.409453 11656 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:46.411230 11603 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.411859 11603 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.414891 11603 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.415199 11629 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.322644 11602 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:46.418426 11644 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.418954 11602 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.380245  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:46.422709 11532 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.423080 11532 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:46.423429 11532 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.426397  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.426626 11658 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.427083  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:46.427115 11658 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.380638  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:46.430711  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50186.2us
I20260709 13:00:46.432713  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.384557  8818 rpcz_store.cc:276] 0709 13:00:45.266297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.654805 (+388508us) service_pool.cc:224] Handling call
0709 13:00:46.365556 (+710751us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:46.365707 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177243008}
I20260709 13:00:46.386559 11474 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.406509  8821 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.376955  8823 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:46.467533 11427 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.407024  8822 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:46.468039  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.483809  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.484256  8824 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:46.484452  8819 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.485172 11417 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.483836 11485 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.485908 11417 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.488626 11417 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.436800 11546 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.490449 11546 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.490521  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.494411 11326 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.495038 11544 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:46.495318  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.495667 11544 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.497200 11624 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:46.498512 11624 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.499264 11604 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.500985 11624 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.503185  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.504220 11612 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:46.504395 11658 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.505062  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:46.505712  9441 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.506510 11595 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.507480 11620 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.508563 11514 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.509440  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.510191  9446 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:46.511231 11349 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.511732 11349 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.512226  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.512517 11326 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.513444 11406 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.514298 11406 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:46.372934 11599 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:46.519129 11615 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:46.519558 11615 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:46.517956 11270 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:46.519977  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.520354 11270 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.520673  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.520884 11270 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.522328  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.523130  8819 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:46.524969  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.525625  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.520680 11615 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.526441  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.526988 11615 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.529637  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:46.530545 11542 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.513520 11644 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.517773 11666 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:00:46.535669  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.469555 11290 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.531735 11421 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:46.504792 11612 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:46.475811 11648 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:46.530557  8823 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:46.536046  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.470248  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.539554  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.540702  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.541780  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.585100  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.542610  9443 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 5 LEARNER]: Advancing to term 7
I20260709 13:00:46.585482  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.544169  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.518682 11599 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.545022  8822 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:46.586565  8741 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:46.586849  9440 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:46.543099 11349 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:46.546074  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.547837 11544 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.515887 11490 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.627640  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.629336  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.537492  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.552340  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.630977  8742 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:46.553390  8818 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:46.554071 11646 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.554467  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.555073  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.559916  8824 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.556695 11592 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.563562 11644 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.513015 11326 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.566727  9441 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:46.649484  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.650373  9441 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:46.568519 11542 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.621210 11595 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.585744  8745 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:46.588335  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:46.652163  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 63940.5us
I20260709 13:00:46.653775  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.655036  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:46.655776  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.516816 11406 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:46.589219  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:46.590478  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.658551  8744 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:46.595237  9445 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:46.549540 11617 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.616480  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.622435  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.661684  9442 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:46.623941  8743 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.634454  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.635027  8747 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:46.634454  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:46.636736  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 99388.3us
I20260709 13:00:46.640076  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:46.643345 11656 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.583695 11620 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.667755  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.666666  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.647436  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.666839  9443 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 7 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:46.659874  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.669379  9440 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:46.670748  9440 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:46.660698  8746 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:46.664350  8820 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.664368  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:46.666532  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.673516  9442 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:00:46.668378  9448 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.646735  8824 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:46.648469 11349 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.618122 11604 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:00:46.651825 11599 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:46.663209 11409 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:46.624454 11519 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:00:46.692008  9441 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:46.751096  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 484418us
I20260709 13:00:46.688968  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.556706 11651 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:46.689034  8643 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 299) took 19691 ms (client timeout 15000 ms). Trace:
I20260709 13:00:46.667917 11547 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:46.668040 11417 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:46.701063  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 400771us
W20260709 13:00:46.762764  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45828us
I20260709 13:00:46.634110  9444 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:46.752341  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 472651us
I20260709 13:00:46.753028  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.756449  9447 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:46.668066 11620 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.763577  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.784596  8820 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:46.768455  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.769181  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.770117  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.788344  8818 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:46.770960  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.789449  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:46.772760  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.790781  8891 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:46.778111  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.693712 11666 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:46.699697 11648 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:46.781669  8823 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.783592  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.703960 11600 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:46.666036 11541 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:00:46.799083  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.800251  9446 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.800820  8894 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:46.821395  8895 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:46.668207 11576 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.665643 11595 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:46.808598  8747 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.810292  8745 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:46.827097  8745 pending_rounds.cc:85] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 2
I20260709 13:00:46.827402  8745 pending_rounds.cc:107] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:46.827682  8745 raft_consensus.cc:2889] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.807199  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:46.813656  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:46.829162  8892 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:46.830068  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.818397  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.830747  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.764487 11547 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.819644  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:46.753979  8643 rpcz_store.cc:269] 0709 13:00:26.997506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.021410 (+ 23904us) service_pool.cc:224] Handling call
0709 13:00:46.688960 (+19667550us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:43.329166 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:43.386264 (+ 57098us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:43.386313 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:00:43.386326 (+    13us) tablet.cc:662] Decoding operations
0709 13:00:43.387200 (+   874us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:43.387283 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:43.387314 (+    31us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:00:43.465351 (+ 78037us) tablet.cc:701] Row locks acquired
0709 13:00:43.465444 (+    93us) write_op.cc:260] PREPARE: finished
0709 13:00:43.465639 (+   195us) write_op.cc:270] Start()
0709 13:00:43.465814 (+   175us) write_op.cc:276] Timestamp: P: 1783602043465567 usec, L: 0
0709 13:00:43.465840 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:00:43.466523 (+   683us) log.cc:844] Serialized 2497 byte log entry
0709 13:00:44.173758 (+707235us) op_driver.cc:464] REPLICATION: finished
0709 13:00:44.392294 (+218536us) write_op.cc:301] APPLY: starting
0709 13:00:44.392433 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:44.392728 (+   295us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:44.392928 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:44.425027 (+ 32099us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:44.488753 (+ 63726us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:00:44.488840 (+    87us) write_op.cc:312] APPLY: finished
0709 13:00:44.489374 (+   534us) log.cc:844] Serialized 53 byte log entry
0709 13:00:44.580294 (+ 90920us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:44.580437 (+   143us) write_op.cc:454] Released schema lock
0709 13:00:44.580519 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:44.580983 (+   464us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:44.581018 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1326442,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":218177,"apply.run_cpu_time_us":15621,"apply.run_wall_time_us":188699,"num_ops":6,"prepare.queue_time_us":239555,"prepare.run_cpu_time_us":40883,"prepare.run_wall_time_us":455649,"raft.queue_time_us":208526,"raft.run_cpu_time_us":34913,"raft.run_wall_time_us":166302,"replication_time_us":707702,"thread_start_us":630792,"threads_started":4,"wal-append.queue_time_us":378775,"wal-append.run_cpu_time_us":13397,"wal-append.run_wall_time_us":1653371}]]}
I20260709 13:00:46.762166 11417 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.698199 11349 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.749315 11531 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:00:46.793866 11620 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.815789  8743 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:46.793493 11542 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.794706 11406 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:00:46.667651 11612 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.807166 11648 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.842849  8815 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.841423 11417 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.843686  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.819442 11600 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.840333  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.836876  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.781318 11537 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:00:46.824512 11290 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.832356  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.819530 11409 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:46.837476 11629 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:00:46.823604 11666 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:46.854142  8788 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:46.900595  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46578.9us
I20260709 13:00:46.901762  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.903101  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.834702 11547 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:46.838186  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.843479 11349 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.704504 11390 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:00:46.835594 11532 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:46.917064  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.870256 11618 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.872579 11664 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:46.847071 11531 consensus_queue.cc:1243] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:46.868036  8819 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:46.877867  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.896217  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.920483  8747 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:46.920687  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.921073  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.921342  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.864437 11542 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.866302 11648 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.943915  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:46.944358  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:46.944921  8819 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:46.907794  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:46.946450  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:46.947916  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:46.864220  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:46.950103  8746 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.842453 11514 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:00:46.921604  8822 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:00:46.922592  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:46.898972 11519 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:46.846625 11662 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:00:46.875149 11600 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:46.928495  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.930859  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:46.873977 11656 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:46.860335 11617 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:00:46.873144 11270 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:46.873653  8646 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 4 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:00:46.955703  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 263684us
I20260709 13:00:46.956912  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:47.005616  9449 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:47.027038  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:46.922550 11449 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:46.998886  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.047838  8746 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:46.924431 11490 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:47.030354  8646 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:47.030632  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73874.5us
I20260709 13:00:46.947389 11649 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.952898 11621 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:46.946651 11290 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.002344  8824 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:46.965400  9444 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:00:46.917335 11544 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:00:46.965005 11624 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:47.006688 11541 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.004258 11546 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.008100 11537 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.051023 11449 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.028085 11326 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:47.013397 11426 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.201195  8646 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.952167 11542 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:47.205819  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 404394us
I20260709 13:00:47.220968  8824 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:47.221457  8824 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:47.221796  8824 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:47.142127  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.216050  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:47.003388 11441 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:00:47.212174 11624 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:47.112529 11656 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:47.110633 11618 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.201653 11651 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:47.217871 11449 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.110977 11290 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.209903 11537 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:47.232999 11677 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 13:00:47.239634  8646 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 4, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:47.212370 11541 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.207187 11621 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:47.268193  8741 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:47.296247  8646 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 1 to 3, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:47.262161 11618 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.118541 11649 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:47.268105 11537 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.263801 11290 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.236442 11326 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:00:47.111910 11409 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:47.386538  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 483569us
I20260709 13:00:47.260855 11624 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:47.384702  8646 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:47.443243  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.298904 11541 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.273968 11390 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.274618 11595 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:47.295926 11576 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:47.486532  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:47.384701 11406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.002s	sys 0.014s
W20260709 13:00:47.490623  8891 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.005s	sys 0.042s
W20260709 13:00:47.516469  8891 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.006s	sys 0.042s
I20260709 13:00:47.495057  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:47.501253  9449 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:47.505788  9445 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:47.202720 11666 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:47.474649 11618 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.508796  8646 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:47.521632  8818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.038s	sys 0.058s
W20260709 13:00:47.522392  8818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.038s	sys 0.058s
W20260709 13:00:47.502533 11474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.022s	sys 0.100s
W20260709 13:00:47.513902  8862 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.004s	sys 0.068s
W20260709 13:00:47.571998  8862 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.005s	sys 0.069s
W20260709 13:00:47.572353  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 796829us
I20260709 13:00:47.574186  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:47.514025  8740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.021s	sys 0.053s
W20260709 13:00:47.575111  8740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.021s	sys 0.053s
W20260709 13:00:47.519243  8712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.007s	sys 0.029s
W20260709 13:00:47.510118  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.017s	sys 0.009s
W20260709 13:00:47.547629  9415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.006s	sys 0.022s
I20260709 13:00:47.570919  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:47.576032  8712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.007s	sys 0.029s
W20260709 13:00:47.579277  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.000s	sys 0.030s
W20260709 13:00:47.581570  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.000s	sys 0.031s
I20260709 13:00:47.304849 11621 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:47.582034  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 552461us
W20260709 13:00:47.576092  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.018s	sys 0.009s
W20260709 13:00:47.626673  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 52598.8us
I20260709 13:00:47.627934  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:47.583827 11474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.023s	sys 0.101s
W20260709 13:00:47.626818  9413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.011s	sys 0.044s
W20260709 13:00:47.633598  9413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.012s	sys 0.044s
I20260709 13:00:47.630045  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:47.585085 11595 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:47.619354 11599 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.004s
W20260709 13:00:47.656145  9415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.024s
W20260709 13:00:47.630530 11606 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.010s	sys 0.019s
I20260709 13:00:47.514664 11390 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:47.611547 11532 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:47.636144 11519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.027s
W20260709 13:00:47.708630  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 778177us
I20260709 13:00:47.635208 11270 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:47.785822 11270 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:47.787184 11270 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.788255 11426 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:47.729144  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:47.788908 11426 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:47.738147 11672 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 13:00:47.825433  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:47.740844 11421 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.146s	user 0.005s	sys 0.004s
I20260709 13:00:47.741741 11687 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:47.725715 11606 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.010s	sys 0.019s
I20260709 13:00:47.748407  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 642) took 1206 ms. Trace:
I20260709 13:00:47.876060  9447 rpcz_store.cc:276] 0709 13:00:46.542326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.581971 (+ 39645us) service_pool.cc:224] Handling call
0709 13:00:47.748132 (+1166161us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:47.748321 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176523008}
I20260709 13:00:47.749120  9441 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:47.749907 11688 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:47.750883 11675 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:00:47.750962  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:47.755301 11678 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:00:47.654083 11406 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.206s
I20260709 13:00:47.757269 11390 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:47.758535 11644 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:47.765368 11519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.009s	sys 0.028s
I20260709 13:00:47.767334 11621 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:47.891522 11673 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.662s	user 0.010s	sys 0.000s
W20260709 13:00:47.769418 11674 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.291s	user 0.000s	sys 0.002s
I20260709 13:00:47.777173  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 426) took 1186 ms. Trace:
W20260709 13:00:47.777666 11686 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:00:47.782189  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 602) took 1118 ms. Trace:
I20260709 13:00:47.736986  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:47.790575  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:47.792487 11485 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:47.795264 11667 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:00:47.796764 11671 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:00:47.798889 11592 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:47.802037 11612 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.802244 11649 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:47.802982 11542 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:47.649513 11441 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:47.635982 11656 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:47.805778 11615 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.278s	user 0.013s	sys 0.000s
I20260709 13:00:47.862711  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:47.972754  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110207us
I20260709 13:00:47.742678 11682 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:47.634490 11629 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:47.884503  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.034s	sys 0.024s
W20260709 13:00:47.980008  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.034s	sys 0.024s
I20260709 13:00:47.885214  9446 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.720217 11599 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.755448  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 600) took 1102 ms. Trace:
I20260709 13:00:47.949121 11576 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:47.898098  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 422247us
I20260709 13:00:47.898547  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:47.987746  8787 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.587s (SENT) [suppressed 15 similar messages]
W20260709 13:00:47.988426  8787 leader_election.cc:336] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.680s (SENT)
I20260709 13:00:47.988838  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:47.987702  9447 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:00:47.907332  8824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.006s	sys 0.020s
W20260709 13:00:47.767726 11668 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:00:47.914070 11688 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:47.913882  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:47.914628  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1403) took 1621 ms. Trace:
I20260709 13:00:47.990336  8817 rpcz_store.cc:276] 0709 13:00:46.293521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.293635 (+   114us) service_pool.cc:224] Handling call
0709 13:00:46.318814 (+ 25179us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:46.319181 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.319214 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:46.319750 (+   536us) log.cc:844] Serialized 52 byte log entry
0709 13:00:46.802974 (+483224us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:46.803013 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.803077 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.914398 (+1111321us) raft_consensus.cc:1685] finished
0709 13:00:47.914418 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.914552 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3322,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":12040,"thread_start_us":441966,"threads_started":1,"wal-append.queue_time_us":943591}
I20260709 13:00:47.972437 11596 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:47.874866 11416 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.923732  9448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.036s	sys 0.027s
W20260709 13:00:48.058200  9448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.037s	sys 0.027s
I20260709 13:00:47.888480 11532 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.922199  8740 rpcz_store.cc:276] 0709 13:00:46.590699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.636775 (+ 46076us) service_pool.cc:224] Handling call
0709 13:00:46.660628 (+ 23853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.665656 (+  5028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.665698 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.666261 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.839292 (+173031us) spinlock_profiling.cc:243] Waited 931 us on lock 0x7b18002dd660. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:00:47.575593 (+736301us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:47.575653 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.575725 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.776806 (+201081us) raft_consensus.cc:1685] finished
0709 13:00:47.776825 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.777091 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4663,"raft.queue_time_us":7410,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10257,"spinlock_wait_cycles":2141952,"thread_start_us":736061,"threads_started":1,"wal-append.queue_time_us":1027245}
W20260709 13:00:47.923909 11349 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.971s	user 0.009s	sys 0.002s
I20260709 13:00:47.907496 11519 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:47.948988  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 158568us
W20260709 13:00:48.067814  8891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.001s	sys 0.047s
W20260709 13:00:48.068472  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.068668  8891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.047s
W20260709 13:00:48.037153 11290 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.002s
W20260709 13:00:47.950152  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.929122 11485 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:47.944671 11602 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.879140  8646 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 reported cstate change: leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 4 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:00:47.981807  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.082922  9446 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:00:47.989756  8824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.006s	sys 0.020s
I20260709 13:00:47.915998  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1718) took 1139 ms. Trace:
I20260709 13:00:47.992838  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1722) took 1210 ms. Trace:
I20260709 13:00:48.086021  8748 rpcz_store.cc:276] 0709 13:00:46.782484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.782868 (+   384us) service_pool.cc:224] Handling call
0709 13:00:46.783163 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.783515 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.783546 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.800553 (+ 17007us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.243105 (+442552us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:47.243157 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.243221 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.992570 (+749349us) raft_consensus.cc:1685] finished
0709 13:00:47.992596 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.992760 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4890,"raft.queue_time_us":514,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":13789,"thread_start_us":442215,"threads_started":1,"wal-append.queue_time_us":827557}
I20260709 13:00:47.950884 11621 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:47.945345 11670 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.002s	sys 0.030s
I20260709 13:00:48.010190  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1732) took 1771 ms. Trace:
W20260709 13:00:48.011206 11620 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.155s	user 0.013s	sys 0.000s
I20260709 13:00:48.084429  8749 rpcz_store.cc:276] 0709 13:00:46.776226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.776356 (+   130us) service_pool.cc:224] Handling call
0709 13:00:46.806585 (+ 30229us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:46.834373 (+ 27788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.834416 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:46.846014 (+ 11598us) log.cc:844] Serialized 52 byte log entry
0709 13:00:47.398687 (+552673us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:47.475945 (+ 77258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.476031 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.915800 (+439769us) raft_consensus.cc:1685] finished
0709 13:00:47.915818 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.915946 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12875,"raft.queue_time_us":18881,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":10090,"thread_start_us":439194,"threads_started":1,"wal-append.queue_time_us":880752}
W20260709 13:00:48.013203 11417 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.136s	user 0.007s	sys 0.007s
I20260709 13:00:48.008874 11624 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.013722  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:48.096043  8893 rpcz_store.cc:276] 0709 13:00:46.238712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.283653 (+ 44941us) service_pool.cc:224] Handling call
0709 13:00:46.283876 (+   223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.284200 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.284236 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.308297 (+ 24061us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.754005 (+445708us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:46.754045 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.754108 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.767042 (+1012934us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:48.009866 (+242824us) raft_consensus.cc:1685] finished
0709 13:00:48.009891 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.010124 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":16145,"spinlock_wait_cycles":9519488,"thread_start_us":396818,"threads_started":1,"wal-append.queue_time_us":912895}
I20260709 13:00:48.104533  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:48.015936  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 605) took 1313 ms. Trace:
I20260709 13:00:48.105036  9447 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:48.017203  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2043) took 1162 ms. Trace:
I20260709 13:00:48.023401 11688 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.024288 11576 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:48.024626 11547 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.078s	user 0.010s	sys 0.001s
I20260709 13:00:47.992866 11599 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:48.032239 11676 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.529s	user 0.000s	sys 0.002s
I20260709 13:00:47.976763 11682 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:48.044548 11514 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.050316 11680 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:48.052529  9412 leader_election.cc:336] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.535s (SENT)
I20260709 13:00:48.129805  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:47.988727 11662 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:48.054210  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1733) took 1815 ms. Trace:
I20260709 13:00:48.062191  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 597) took 1416 ms. Trace:
W20260709 13:00:48.133623  9440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.000s	sys 0.021s
W20260709 13:00:48.133978  9440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.000s	sys 0.021s
I20260709 13:00:48.133795  9448 rpcz_store.cc:276] 0709 13:00:46.646001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.691537 (+ 45536us) service_pool.cc:224] Handling call
0709 13:00:46.704267 (+ 12730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.704475 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:47.227449 (+522974us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.061825 (+834376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.061890 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:48.061934 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.062007 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.062135 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47115,"prepare.queue_time_us":1042762,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":6835,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3951,"thread_start_us":1265100,"threads_started":2,"wal-append.queue_time_us":902063}
I20260709 13:00:48.136307  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 579) took 1273 ms. Trace:
I20260709 13:00:47.980671 11542 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:00:48.065508  8815 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.037s	sys 0.022s
W20260709 13:00:48.040989 11541 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.001s
W20260709 13:00:47.949476  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 105926us
W20260709 13:00:48.077396  9442 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.254s	user 0.016s	sys 0.040s
I20260709 13:00:48.062618 11532 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:48.140151  8815 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.037s	sys 0.022s
I20260709 13:00:48.058928 11615 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:48.104153 11621 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:47.985215  9445 rpcz_store.cc:276] 0709 13:00:46.652933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.752500 (+ 99567us) service_pool.cc:224] Handling call
0709 13:00:47.501778 (+749278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.519425 (+ 17647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.755211 (+235786us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54006801d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:47.755382 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1723149184}
I20260709 13:00:48.101581  8646 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 3 to 2, leader changed from <none> to 112105225cc140459688be5d31dcb071 (127.7.117.132), 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from VOTER to NON_VOTER, now has a pending config: VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129), VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130), VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131), VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:48.091007  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:47.933491  9444 rpcz_store.cc:276] 0709 13:00:46.663365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.825788 (+162423us) service_pool.cc:224] Handling call
0709 13:00:46.877015 (+ 51227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.119111 (+242096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.781843 (+662732us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b54006dfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:47.782104 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55110912}
I20260709 13:00:47.972203 11592 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:00:47.973379 11656 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:47.972592 11441 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:48.098556  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.104578 11417 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:48.011832  8787 leader_election.cc:336] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.706s (SENT)
I20260709 13:00:48.104167 11620 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.014767 11679 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:00:48.105630  8818 rpcz_store.cc:276] 0709 13:00:46.854804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.855019 (+   215us) service_pool.cc:224] Handling call
0709 13:00:46.855356 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.866772 (+ 11416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.866813 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.867319 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.658337 (+791018us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:47.708087 (+ 49750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.708171 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.016990 (+308819us) raft_consensus.cc:1685] finished
0709 13:00:48.017011 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.017150 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10914,"raft.queue_time_us":11062,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":5448,"thread_start_us":575163,"threads_started":1,"wal-append.queue_time_us":1062016}
I20260709 13:00:48.105189  9441 rpcz_store.cc:276] 0709 13:00:46.702777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.828776 (+125999us) service_pool.cc:224] Handling call
0709 13:00:47.748888 (+920112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.877696 (+128808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.015659 (+137963us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54004d1050. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:48.015858 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075559808}
I20260709 13:00:48.114527  8816 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:48.139933 11542 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:48.105264  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 475489us
I20260709 13:00:48.193691  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:48.193495  8646 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 4, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 4 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:48.194562  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:48.126405 11603 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 13:00:48.119102  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 490126us
W20260709 13:00:48.095968 11670 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.002s	sys 0.030s
W20260709 13:00:48.130785  8742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.011s	sys 0.009s
W20260709 13:00:48.229722  8742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.011s	sys 0.009s
I20260709 13:00:48.069902 11349 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.239568  8743 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:48.119400 11427 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:48.138370  8864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.008s	sys 0.013s
W20260709 13:00:48.243935  8864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.008s	sys 0.013s
W20260709 13:00:48.244285  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 1.02151e+06us
I20260709 13:00:48.136607  9448 rpcz_store.cc:276] 0709 13:00:46.862581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.135588 (+1273007us) service_pool.cc:224] Handling call
0709 13:00:48.135857 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.136016 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.136040 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.136077 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:48.136113 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.136162 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.136259 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.986841 11390 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:48.121091 11576 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:48.140522  9442 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.327s	user 0.017s	sys 0.040s
I20260709 13:00:48.133630  8889 rpcz_store.cc:276] 0709 13:00:46.238960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.285521 (+ 46561us) service_pool.cc:224] Handling call
0709 13:00:46.285801 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:46.340493 (+ 54692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.340521 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:46.345516 (+  4995us) log.cc:844] Serialized 52 byte log entry
0709 13:00:46.510917 (+165401us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b180002d340. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:46.695372 (+184455us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:46.695429 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.695516 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.768681 (+1073165us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:48.053905 (+285224us) raft_consensus.cc:1685] finished
0709 13:00:48.053940 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.054134 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51444,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":19278848,"thread_start_us":374115,"threads_started":2,"wal-append.queue_time_us":889568}
I20260709 13:00:48.143607  8745 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:48.141434  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 88998.1us
I20260709 13:00:48.141891  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:48.249752  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 108013us
I20260709 13:00:48.146019  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 598) took 1499 ms. Trace:
I20260709 13:00:48.065699 11416 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.148362  9446 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:48.008874 11426 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.275414  8821 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:00:48.275910  8821 pending_rounds.cc:85] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:48.276163  8821 pending_rounds.cc:107] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:48.276429  8821 raft_consensus.cc:2889] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:48.102625 11617 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:00:48.098860 11290 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.120093 11703 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:48.097905 11618 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.100875 11602 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:48.138684 11669 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.003s	sys 0.017s
I20260709 13:00:48.129554 11680 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:48.183807  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:48.103941  9449 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.015s	sys 0.035s
W20260709 13:00:48.336773  9449 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.016s	sys 0.035s
I20260709 13:00:48.146752 11532 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.068430 11596 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:48.133101 11514 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.103044 11648 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.128s	user 0.005s	sys 0.005s
I20260709 13:00:48.197129 11542 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:48.197788  8646 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.200078  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.200795  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.201442 11615 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.202209 11701 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:48.202905 11441 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.202977  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.205193 11406 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:48.206017 11592 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:48.206643 11700 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:48.208231 11326 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:48.211378  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:48.214573 11600 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.177s	user 0.008s	sys 0.003s
I20260709 13:00:48.219944 11599 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.222189  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 594) took 1588 ms. Trace:
I20260709 13:00:48.166451 11485 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.126935 11547 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:48.195171  8818 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:48.426582  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:48.229934  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:48.241277  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1717) took 1465 ms. Trace:
I20260709 13:00:48.429597  8742 rpcz_store.cc:276] 0709 13:00:46.775918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.776657 (+   739us) service_pool.cc:224] Handling call
0709 13:00:46.799407 (+ 22750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.799651 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:47.261658 (+462007us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.240883 (+979225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.240950 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:48.241015 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.241097 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.241216 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58892,"prepare.queue_time_us":44892,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4189,"raft.queue_time_us":11513,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":32355,"thread_start_us":1405447,"threads_started":2}
I20260709 13:00:48.239799  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 464) took 1458 ms. Trace:
I20260709 13:00:48.146533 11541 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.250579  9440 rpcz_store.cc:276] 0709 13:00:46.646263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.690152 (+ 43889us) service_pool.cc:224] Handling call
0709 13:00:46.690509 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.690752 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:47.084863 (+394111us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.145595 (+1060732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.145661 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:48.145715 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.145797 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.145951 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6629,"prepare.queue_time_us":935886,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":45,"raft.queue_time_us":10780,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"thread_start_us":1381372,"threads_started":2}
W20260709 13:00:48.252453  8741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.015s	sys 0.023s
W20260709 13:00:48.434038  8741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.016s	sys 0.023s
I20260709 13:00:48.247493  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:48.282291  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.142606 11662 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.319917  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:48.340211  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 466389us
I20260709 13:00:48.346442  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1423) took 1579 ms. Trace:
W20260709 13:00:48.434927  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 187619us
I20260709 13:00:48.348744  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 608) took 1498 ms. Trace:
W20260709 13:00:48.435341  8712 leader_election.cc:336] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.625s (SENT)
I20260709 13:00:48.435794  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:48.273880 11537 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.775s	user 0.011s	sys 0.000s
I20260709 13:00:48.431175  8891 rpcz_store.cc:276] 0709 13:00:46.797972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.798088 (+   116us) service_pool.cc:224] Handling call
0709 13:00:46.798469 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.516850 (+718381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.516887 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.517432 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.069230 (+551798us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:48.069311 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.069382 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.239536 (+170154us) raft_consensus.cc:1685] finished
0709 13:00:48.239564 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.239731 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":565,"raft.queue_time_us":323748,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":14698,"thread_start_us":1269410,"threads_started":2,"wal-append.queue_time_us":719051}
I20260709 13:00:48.356483  9444 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:00:48.335875 11669 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.214s	user 0.003s	sys 0.017s
I20260709 13:00:48.398396  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.394235 11705 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:48.186439 11519 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.408008 11710 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:00:48.403937 11615 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.401503 11701 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.253309 11427 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.351572 11658 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:48.477919 11441 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.478525 11441 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.479197 11514 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.414815  8646 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 4 to 5, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 5 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.289875 11390 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.243945 11682 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.422864  9443 rpcz_store.cc:276] 0709 13:00:46.634139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.669509 (+ 35370us) service_pool.cc:224] Handling call
0709 13:00:46.669772 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.670102 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.670128 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.670529 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.968315 (+297786us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:46.968370 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.968467 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.221947 (+1253480us) raft_consensus.cc:1685] finished
0709 13:00:48.221971 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.222122 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19952,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9585,"thread_start_us":297485,"threads_started":1,"wal-append.queue_time_us":1218318}
I20260709 13:00:48.425026  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.491789  8749 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:48.492969  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.436295  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:48.494540  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58363.7us
I20260709 13:00:48.495342 11349 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:48.495930  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:48.496083 11349 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:48.496659 11326 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:48.496877 11349 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:48.435585  9447 rpcz_store.cc:276] 0709 13:00:46.850873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.115225 (+1264352us) service_pool.cc:224] Handling call
0709 13:00:48.127990 (+ 12765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.128239 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:48.142525 (+ 14286us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.348245 (+205720us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.348413 (+   168us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:48.348488 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.348565 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.348686 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15784,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":2027,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":43344,"thread_start_us":202501,"threads_started":1,"wal-append.queue_time_us":282285}
I20260709 13:00:48.435220  8821 rpcz_store.cc:276] 0709 13:00:46.767177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.769730 (+  2553us) service_pool.cc:224] Handling call
0709 13:00:46.806558 (+ 36828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.276904 (+1470346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.346187 (+ 69283us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400020dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:48.346384 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3378587392}
I20260709 13:00:48.441350 11600 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:48.515362 11733 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:00:48.433430 11700 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.446195 11547 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.412422 11592 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:48.473863 11701 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.446735 11621 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:48.446624 11683 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:00:48.562139 11683 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:48.562613 11683 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:48.563508 11683 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.564770 11683 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.446954 11485 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.447635 11728 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:48.565668 11728 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.566126 11728 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.448130 11537 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.449009 11696 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:00:48.449438 11644 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:48.449967 11474 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:48.583096 11474 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:48.583446 11474 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:48.450881  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 221138us
I20260709 13:00:48.585860  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:48.514099 11349 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.586575 11600 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.451475 11670 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:48.586864 11349 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.446254 11417 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.448143  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.481769  8646 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 3 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.589378  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.449033  8748 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:48.341150 11703 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.459095 11705 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.471506  8746 raft_consensus.cc:2368] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8.
I20260709 13:00:48.466714 11599 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:48.477191 11615 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.477095  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.394546 11603 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:48.608363 11603 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.438130 11612 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.497262 11326 consensus_queue.cc:1243] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:48.610690 11612 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:48.497995 11449 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.224s	user 0.009s	sys 0.005s
I20260709 13:00:48.611912 11449 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.467368 11708 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:00:48.468773 11617 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:48.501387  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.501965  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1733) took 1237 ms. Trace:
I20260709 13:00:48.305384 11595 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:48.502748  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.502843 11662 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.504925 11602 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.613919  8820 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:48.614662  8743 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.622994  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:48.623613  8816 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.510126  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:48.461813 11624 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:48.397598 11648 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.613561 11705 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.649813 11648 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.474067 11253 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.533598 11618 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.534660 11426 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:48.651376  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 64424.9us
W20260709 13:00:48.521015 11712 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:00:48.539072 11656 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.539273 11531 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:48.652873 11531 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:48.536216 11541 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:48.540930 11290 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:48.474391 11596 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.653739 11531 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.648941 11490 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:48.654700 11490 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:48.654978 11531 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:48.655071 11490 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.540123 11682 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.551635  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2047) took 1331 ms. Trace:
I20260709 13:00:48.553498 11592 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.557049  8747 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:48.558408 11427 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.423501 11680 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.657017  8824 rpcz_store.cc:276] 0709 13:00:47.220061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.220245 (+   184us) service_pool.cc:224] Handling call
0709 13:00:47.220707 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:47.233542 (+ 12835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.233599 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:47.234259 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 13:00:48.084482 (+850223us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:48.084521 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.084599 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.551361 (+466762us) raft_consensus.cc:1685] finished
0709 13:00:48.551388 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.551563 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49939,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":808687,"threads_started":1,"wal-append.queue_time_us":856177}
I20260709 13:00:48.668907  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:48.669584  8824 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.531409 11621 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.697902 11621 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.700484 11621 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:48.566813 11666 consensus_queue.cc:1175] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "29c7934ee528454785432732d01ed0a0" responder_term: 7 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:00:48.577014  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.702795  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:48.579955  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.703935 11718 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:48.704474 11718 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.704739 11624 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.705561 11624 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.714438  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:48.716632  8749 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.581768  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:48.598111  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.602259 11644 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:48.606477 11732 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.083s	user 0.004s	sys 0.000s
W20260709 13:00:48.628115  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 118111us
I20260709 13:00:48.650509 11253 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:48.654240 11449 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:48.529747 11620 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:48.556346 11700 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.584262 11474 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.625212  8741 rpcz_store.cc:276] 0709 13:00:47.264633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.276744 (+ 12111us) service_pool.cc:224] Handling call
0709 13:00:47.277119 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.300313 (+ 23194us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:47.479638 (+179325us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.010510 (+530872us) spinlock_profiling.cc:243] Waited 7.58 ms on lock 0x7b18004aeae0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:48.449392 (+438882us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.449858 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.450005 (+   147us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:48.450038 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.450110 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.501653 (+ 51543us) raft_consensus.cc:1685] finished
0709 13:00:48.501770 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.501905 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14876,"prepare.queue_time_us":179092,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":75534,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":81266,"spinlock_wait_cycles":17442688,"thread_start_us":851592,"threads_started":1,"wal-append.queue_time_us":885351}
I20260709 13:00:48.556644 11576 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.678706 11595 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:48.613159 11612 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:48.704860 11718 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.590576  8646 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.652639 11542 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:49.229024 11682 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.001s
W20260709 13:00:48.794037  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.755739 11290 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:48.924841 11704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.016s	sys 0.054s
I20260709 13:00:49.325997 11290 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.820284  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.238758 11670 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
I20260709 13:00:49.327255 11670 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:49.327435  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.184324  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.762862 11592 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:49.190323  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.924453  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.021s
W20260709 13:00:49.168359  8743 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
W20260709 13:00:49.137528  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.063005  9449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.015s	sys 0.020s
I20260709 13:00:49.331209  8743 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:00:49.331377  9449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.020s	sys 0.025s
I20260709 13:00:49.325170 11680 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.332062 11658 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:48.903574  8822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.035s	sys 0.061s
W20260709 13:00:49.333815  8822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.040s	sys 0.070s
I20260709 13:00:48.789817  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1730) took 1964 ms. Trace:
W20260709 13:00:48.915673  8815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.009s	sys 0.040s
W20260709 13:00:49.335165  8815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.011s	sys 0.053s
I20260709 13:00:49.168398  8824 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:49.324465  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 461) took 2550 ms. Trace:
W20260709 13:00:49.325438  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.323962  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:48.693162  8816 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
W20260709 13:00:48.923799 11421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.014s	sys 0.055s
I20260709 13:00:49.338929  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:49.332989  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:48.838317 11532 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:48.923194  8817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.027s	sys 0.043s
I20260709 13:00:49.325135 11682 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:49.330718  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.028s	sys 0.027s
I20260709 13:00:49.349892  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 607) took 2516 ms. Trace:
W20260709 13:00:48.920495  9412 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.015s	sys 0.024s
W20260709 13:00:49.810438  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.465s	user 0.005s	sys 0.031s
W20260709 13:00:50.469449  8817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.286s	user 0.046s	sys 0.062s
I20260709 13:00:50.464071  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2046) took 3557 ms. Trace:
I20260709 13:00:49.359213  8821 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:49.357980  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:50.470010  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.130s	user 0.005s	sys 0.041s
W20260709 13:00:50.470968  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2056) took 1849 ms (client timeout 2016 ms). Trace:
I20260709 13:00:50.471370  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2048) took 2392 ms. Trace:
W20260709 13:00:50.471669  8821 rpcz_store.cc:269] 0709 13:00:48.621825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.622655 (+   830us) service_pool.cc:224] Handling call
0709 13:00:50.470674 (+1848019us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.532274  8818 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.010s	sys 0.021s
W20260709 13:00:50.473932  8818 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.015s	user 0.017s	sys 0.032s
I20260709 13:00:50.474027  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:50.474967  8821 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:49.443836  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 458) took 1373 ms. Trace:
I20260709 13:00:50.475703  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2061) took 1767 ms. Trace:
I20260709 13:00:50.476107  8821 rpcz_store.cc:276] 0709 13:00:48.707865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.473472 (+1765607us) service_pool.cc:224] Handling call
0709 13:00:50.475616 (+  2144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.475795  8822 rpcz_store.cc:276] 0709 13:00:48.069959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.070084 (+   125us) service_pool.cc:224] Handling call
0709 13:00:48.070376 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.070599 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:48.099522 (+ 28923us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.443215 (+1343693us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.443384 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.443435 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:49.443493 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.443593 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.443762 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108868,"prepare.queue_time_us":29226,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2205,"raft.queue_time_us":112759,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":970,"thread_start_us":1234441,"threads_started":1,"wal-append.queue_time_us":1544443}
W20260709 13:00:50.477959  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.004s	sys 0.007s
W20260709 13:00:50.478494  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.004s	sys 0.007s
W20260709 13:00:49.621258  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 298437us
W20260709 13:00:49.627822  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 296422us
I20260709 13:00:50.480577  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2062) took 1772 ms. Trace:
I20260709 13:00:50.481309  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:50.481585  8861 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.007s	sys 0.014s
W20260709 13:00:50.482100  8861 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.007s	sys 0.014s
I20260709 13:00:49.353768  8816 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.477376  8815 rpcz_store.cc:276] 0709 13:00:46.906232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.906389 (+   157us) service_pool.cc:224] Handling call
0709 13:00:46.906689 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.142526 (+1235837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.142555 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.168074 (+ 25519us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.335794 (+1167720us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.335849 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.335942 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.463306 (+1127364us) raft_consensus.cc:1685] finished
0709 13:00:50.463386 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.463773 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85167,"raft.run_cpu_time_us":3747,"raft.run_wall_time_us":12769,"thread_start_us":2332387,"threads_started":2,"wal-append.queue_time_us":1440002}
W20260709 13:00:50.483760  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.484251  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.470958  8824 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:50.485924  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.496414  8819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.007s	sys 0.023s
W20260709 13:00:50.486826  8819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.171s	user 0.012s	sys 0.030s
I20260709 13:00:50.471817  8817 rpcz_store.cc:276] 0709 13:00:48.079608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.081146 (+  1538us) service_pool.cc:224] Handling call
0709 13:00:48.115718 (+ 34572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:48.116147 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.116178 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:48.118323 (+  2145us) log.cc:844] Serialized 52 byte log entry
0709 13:00:50.470674 (+2352351us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.470827 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.470994 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.471033 (+    39us) raft_consensus.cc:1685] finished
0709 13:00:50.471047 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.471312 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4786,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3674,"thread_start_us":2287234,"threads_started":1,"wal-append.queue_time_us":1498352}
W20260709 13:00:49.855862  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.539s	user 0.012s	sys 0.008s
W20260709 13:00:50.420797  8740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.806s	user 0.000s	sys 0.022s
W20260709 13:00:50.490160  8740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.876s	user 0.000s	sys 0.035s
W20260709 13:00:50.491400  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 2.48043e+06us
W20260709 13:00:49.495453  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.009s	sys 0.021s
W20260709 13:00:50.412189  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.00984e+06us
W20260709 13:00:50.420842  8745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.968s	user 0.010s	sys 0.012s
I20260709 13:00:50.481127  8822 rpcz_store.cc:276] 0709 13:00:48.708184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.478991 (+1770807us) service_pool.cc:224] Handling call
0709 13:00:50.479524 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.480131 (+   607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.480171 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.480243 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:50.480280 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.480351 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.480474 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.422120 11349 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.835s	user 0.012s	sys 0.000s
I20260709 13:00:49.328569 11290 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:50.497344 11349 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:50.489478  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.173s	user 0.012s	sys 0.015s
I20260709 13:00:50.504169 11772 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:50.508947  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:50.513484 11773 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.515182  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2055) took 2276 ms. Trace:
I20260709 13:00:50.516111 11771 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.517210 11772 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.515918  8818 rpcz_store.cc:276] 0709 13:00:48.238653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.457893 (+219240us) service_pool.cc:224] Handling call
0709 13:00:48.458202 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.458408 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:50.474543 (+2016135us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.514394 (+ 39851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.514494 (+   100us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.514643 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.514781 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.515060 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15287,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"spinlock_wait_cycles":13056,"thread_start_us":2054561,"threads_started":3,"wal-append.queue_time_us":31232}
I20260709 13:00:50.518222 11773 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.520047 11774 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:50.520092  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.501096 11349 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:48.731639 11441 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:49.496475 11490 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.000s
W20260709 13:00:50.236847 11514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.756s	user 0.008s	sys 0.008s
I20260709 13:00:50.527493 11474 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:50.236625 11519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.729s	user 0.006s	sys 0.018s
W20260709 13:00:49.333494 11485 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.000s
W20260709 13:00:49.396577 11531 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.741s	user 0.006s	sys 0.003s
I20260709 13:00:50.531886 11532 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.533653 11542 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:49.450311 11576 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.147886 11592 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.818s	user 0.010s	sys 0.000s
W20260709 13:00:50.237493 11595 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.002s	sys 0.010s
I20260709 13:00:49.357532 11605 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:49.353269 11644 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:49.855671 11649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.385s	user 0.008s	sys 0.013s
I20260709 13:00:49.332823 11658 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:50.006690 11669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.394s	user 0.004s	sys 0.016s
W20260709 13:00:49.754235 11695 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.420s	user 0.019s	sys 0.034s
I20260709 13:00:49.776155 11700 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:50.566519  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2059) took 1833 ms. Trace:
I20260709 13:00:50.573254  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 456) took 2837 ms. Trace:
W20260709 13:00:50.413022  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.626039 11519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.106s	user 0.007s	sys 0.019s
W20260709 13:00:50.582998  8745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.130s	user 0.017s	sys 0.019s
I20260709 13:00:49.334443  8747 rpcz_store.cc:276] 0709 13:00:46.825609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.119826 (+294217us) service_pool.cc:224] Handling call
0709 13:00:48.789485 (+1669659us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54004669d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:48.789712 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2970742144}
I20260709 13:00:49.336719  8898 rpcz_store.cc:276] 0709 13:00:46.773640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.773792 (+   152us) service_pool.cc:224] Handling call
0709 13:00:46.774193 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.263261 (+489068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.263292 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.272512 (+  9220us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.945543 (+673031us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:47.945577 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.945642 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.652601 (+706959us) raft_consensus.cc:1685] finished
0709 13:00:48.652627 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.324382 (+671755us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16828,"raft.queue_time_us":67306,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":2525,"thread_start_us":955700,"threads_started":2,"wal-append.queue_time_us":830074}
W20260709 13:00:48.788101 11722 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.238s	user 0.000s	sys 0.002s
W20260709 13:00:49.326567 11704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.129s	user 0.020s	sys 0.066s
W20260709 13:00:49.218428 11701 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
W20260709 13:00:49.497785  9442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.022s	sys 0.007s
W20260709 13:00:49.757212 11629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.219s	user 0.010s	sys 0.034s
W20260709 13:00:49.855361  9445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.426s	user 0.012s	sys 0.008s
W20260709 13:00:49.384895  8749 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.001s
W20260709 13:00:50.166898 11596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.496s	user 0.016s	sys 0.052s
W20260709 13:00:49.496958 11683 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.932s	user 0.013s	sys 0.000s
W20260709 13:00:49.386730 11421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.020s	sys 0.067s
W20260709 13:00:50.236972 11706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.722s	user 0.006s	sys 0.019s
I20260709 13:00:50.625403  8821 rpcz_store.cc:276] 0709 13:00:48.659293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.477048 (+1817755us) service_pool.cc:224] Handling call
0709 13:00:50.488593 (+ 11545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.489018 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.489042 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.489110 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.489140 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.489199 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.492509 (+  3310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.495159 11718 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.628321 11700 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:50.624310  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.213s	user 0.015s	sys 0.028s
W20260709 13:00:50.625738 11695 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.291s	user 0.025s	sys 0.067s
I20260709 13:00:50.620786 11644 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:50.624334 11669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.163s	user 0.005s	sys 0.020s
I20260709 13:00:49.421113  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:50.670655  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 189853us
I20260709 13:00:50.618425 11762 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.625808  8823 rpcz_store.cc:276] 0709 13:00:47.736183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.753939 (+ 17756us) service_pool.cc:224] Handling call
0709 13:00:47.754310 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.238491 (+484181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.238521 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.238929 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.470969 (+2232040us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:50.471094 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.471375 (+   281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.572899 (+101524us) raft_consensus.cc:1685] finished
0709 13:00:50.572921 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.573187 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179047,"raft.queue_time_us":191740,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4800,"thread_start_us":2435439,"threads_started":2,"wal-append.queue_time_us":2290301}
W20260709 13:00:50.660801  8715 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.281s	user 0.016s	sys 0.046s
W20260709 13:00:50.660730  8714 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.289s	user 0.018s	sys 0.009s
W20260709 13:00:50.665433  9412 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.371s	user 0.040s	sys 0.071s
I20260709 13:00:50.668284 11774 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.690289 11762 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.692523 11774 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.693181  8715 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.016s	sys 0.046s
I20260709 13:00:50.697821 11290 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.698594 11531 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.699178 11531 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.698673 11290 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.699527 11531 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.458561 11603 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.701422 11490 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.702191 11603 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.702919 11603 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.704198 11490 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.696586  8714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.019s	sys 0.010s
W20260709 13:00:50.706681  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.710002 11766 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.710569  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:50.710762 11766 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.711534  8747 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:50.712203  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 433) took 2248 ms (client timeout 2888 ms). Trace:
W20260709 13:00:50.712572  8747 rpcz_store.cc:269] 0709 13:00:48.463437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.709891 (+2246454us) service_pool.cc:224] Handling call
0709 13:00:50.712022 (+  2131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.713729 11766 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.717495 11766 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.718237 11766 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.720552 11766 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.662528  8742 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:00:50.721796  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 436) took 2153 ms (client timeout 2126 ms). Trace:
W20260709 13:00:50.722292  8742 rpcz_store.cc:269] 0709 13:00:48.567895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.332737 (+764842us) service_pool.cc:224] Handling call
0709 13:00:50.721669 (+1388932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.668519 11441 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.723801 11441 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.725457 11603 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.654630 11771 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.727306 11441 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.728547  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 2.14771e+06us
I20260709 13:00:50.729499 11771 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:50.732290  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.732025  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:50.733072  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.662856 11773 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.682368 11772 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:50.736583 11649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.156s	user 0.008s	sys 0.017s
W20260709 13:00:50.736918 11595 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.269s	user 0.003s	sys 0.011s
I20260709 13:00:49.408077 11670 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 1.330s
W20260709 13:00:50.737169 11514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.137s	user 0.008s	sys 0.011s
W20260709 13:00:50.713719  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 205280us
W20260709 13:00:50.741202  8789 connection.cc:256] RPC call timeout handler was delayed by -1.6998745020000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.284s for a response
W20260709 13:00:50.752728  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2161063329999706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
I20260709 13:00:50.740845 11773 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.625844  9449 rpcz_store.cc:276] 0709 13:00:46.833226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.518622 (+685396us) service_pool.cc:224] Handling call
0709 13:00:47.518898 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.519107 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:48.337165 (+818058us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.331803 (+994638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.331879 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:49.331943 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.349627 (+ 17684us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.349818 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228500,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":2352,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":58794,"thread_start_us":1749495,"threads_started":2,"wal-append.queue_time_us":1264018,"wal-append.run_cpu_time_us":1451,"wal-append.run_wall_time_us":1010249}
I20260709 13:00:50.663311 11701 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.759039 11777 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:50.749326 11762 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.758260 11290 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:50.772326  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86146.6us
W20260709 13:00:50.769905  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 2.52181e+06us
W20260709 13:00:50.777266  9442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.409s	user 0.028s	sys 0.013s
W20260709 13:00:50.777323  9445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.254s	user 0.014s	sys 0.014s
I20260709 13:00:50.770771 11349 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.803946  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:50.805621  8816 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
W20260709 13:00:50.806628  8816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2063) took 2098 ms (client timeout 1563 ms). Trace:
W20260709 13:00:50.807063  8816 rpcz_store.cc:269] 0709 13:00:48.708441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.338656 (+630215us) service_pool.cc:224] Handling call
0709 13:00:50.806465 (+1467809us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.808363 11592 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.809424 11592 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.811861 11768 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.774085  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:50.814467 11772 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.815043  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 435) took 2201 ms (client timeout 1848 ms). Trace:
I20260709 13:00:50.772791 11774 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:50.776197 11596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.000s	user 0.018s	sys 0.066s
I20260709 13:00:50.816891 11763 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:00:50.818178 11700 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:50.843222 11680 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.436s	user 0.000s	sys 0.011s
I20260709 13:00:50.818670  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:00:50.821655  8788 connection.cc:256] RPC call timeout handler was delayed by -1.4312305849999802s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 13:00:50.824523 11531 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.825439 11767 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.826716 11649 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.827440 11595 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.827574 11416 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.829002 11537 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.829712  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 596) took 4189 ms. Trace:
I20260709 13:00:48.736776 11728 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.851172 11701 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:50.850680  9442 rpcz_store.cc:276] 0709 13:00:46.645706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.674242 (+ 28536us) service_pool.cc:224] Handling call
0709 13:00:46.702162 (+ 27920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.702372 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:48.272851 (+1570479us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.828816 (+2555965us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.828919 (+   103us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.829121 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.829236 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.829550 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55472,"prepare.queue_time_us":996331,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":19107,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":104,"thread_start_us":3946809,"threads_started":2,"wal-append.queue_time_us":2394876}
I20260709 13:00:50.833796  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:50.854159  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 105014us
W20260709 13:00:50.854435  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82860.2us
I20260709 13:00:50.855494  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:50.815450  8744 rpcz_store.cc:269] 0709 13:00:48.567632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.608055 (+ 40423us) service_pool.cc:224] Handling call
0709 13:00:50.765346 (+2157291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.859369  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 434) took 2395 ms. Trace:
I20260709 13:00:50.859874  8744 rpcz_store.cc:276] 0709 13:00:48.463674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.857972 (+2394298us) service_pool.cc:224] Handling call
0709 13:00:50.858508 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.858917 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.858944 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.859019 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:50.859063 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.859131 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.859270 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.784487 11773 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:50.840904  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.778604 11629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.135s	user 0.019s	sys 0.065s
W20260709 13:00:50.784192 11706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.171s	user 0.006s	sys 0.024s
I20260709 13:00:50.767448 11777 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.871927  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 455) took 3142 ms. Trace:
I20260709 13:00:50.871996 11718 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:50.868880 11485 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.873595 11777 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.873379 11774 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.870396 11773 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.875049 11441 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.875664 11441 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:50.876042 11774 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.876974  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1460) took 2296 ms (client timeout 2035 ms). Trace:
W20260709 13:00:50.863463  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88805.8us
W20260709 13:00:50.862654  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111263us
I20260709 13:00:50.816509 11349 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:50.877511  8749 rpcz_store.cc:269] 0709 13:00:48.581367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.623937 (+ 42570us) service_pool.cc:224] Handling call
0709 13:00:50.876868 (+2252931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.861707 11695 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.882980  8749 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:50.883726  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1459) took 2303 ms. Trace:
I20260709 13:00:50.872511  8819 rpcz_store.cc:276] 0709 13:00:47.729532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.729663 (+   131us) service_pool.cc:224] Handling call
0709 13:00:47.729949 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.018228 (+288279us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b48002826d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:00:48.253708 (+235480us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:48.315208 (+ 61500us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.771164 (+2455956us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.825727 (+ 54563us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.826021 (+   294us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:50.826062 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.826164 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.871496 (+ 45332us) raft_consensus.cc:1685] finished
0709 13:00:50.871625 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.871865 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203334,"prepare.queue_time_us":81235,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":249810,"raft.run_cpu_time_us":1517,"raft.run_wall_time_us":74312,"spinlock_wait_cycles":14920960,"thread_start_us":2743925,"threads_started":3,"wal-append.queue_time_us":2258438}
I20260709 13:00:50.885063 11596 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:50.885917 11596 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.886381 11531 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.887972 11767 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:50.889516 11624 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.889746 11767 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:00:50.892669 11742 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:00:50.884039  8749 rpcz_store.cc:276] 0709 13:00:48.580108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.882301 (+2302193us) service_pool.cc:224] Handling call
0709 13:00:50.882686 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.883435 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.883675 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.881546  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:50.784011 11764 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:50.877640 11728 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.901531 11728 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.876813 11441 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.902350 11728 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.784368 11683 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:50.900342 11596 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:50.900671  8787 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.770s (SENT)
I20260709 13:00:50.906374 11728 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.907274 11592 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.907965 11595 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.930310  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 609) took 2944 ms. Trace:
I20260709 13:00:50.931054 11703 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:50.932402 11595 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:50.911139 11605 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.933524 11605 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.935986 11768 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.873055 11718 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.938009  8744 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:50.938689  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1458) took 2362 ms. Trace:
I20260709 13:00:50.939021  8744 rpcz_store.cc:276] 0709 13:00:48.579775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.861126 (+2281351us) service_pool.cc:224] Handling call
0709 13:00:50.895781 (+ 34655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.938404 (+ 42623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.938612 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.940129  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45776.7us
W20260709 13:00:50.918807  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2060) took 2248 ms (client timeout 1576 ms). Trace:
I20260709 13:00:50.921942  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:50.923553  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:50.924252  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:50.925304 11541 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:50.925787  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:50.926285 11649 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.926644  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:50.927744  8822 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.895167 11349 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.927453 11603 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:00:50.928292  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:50.928530  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.905315 11764 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:50.958645 11764 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.959952 11764 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.950161  8817 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:50.941090  8824 rpcz_store.cc:269] 0709 13:00:48.670713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.357660 (+686947us) service_pool.cc:224] Handling call
0709 13:00:50.905733 (+1548073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.942195  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:50.948334  8816 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
W20260709 13:00:50.950318  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49758.4us
I20260709 13:00:50.950526 11656 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:50.951619 11541 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:50.952772  8819 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:50.954033 11603 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:50.930819  9445 rpcz_store.cc:276] 0709 13:00:47.986190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.147497 (+161307us) service_pool.cc:224] Handling call
0709 13:00:48.174962 (+ 27465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.175387 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.175428 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.188432 (+ 13004us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.429225 (+240793us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b18003a3300. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:00:50.898338 (+2469113us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:50.920060 (+ 21722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.929872 (+  9812us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.929923 (+    51us) raft_consensus.cc:1685] finished
0709 13:00:50.929945 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.930245 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":623,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5507,"spinlock_wait_cycles":16558336,"thread_start_us":2376099,"threads_started":1,"wal-append.queue_time_us":2438440}
W20260709 13:00:50.968264  8787 connection.cc:256] RPC call timeout handler was delayed by -2.5996200090000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2457s for a response
I20260709 13:00:50.973774 11703 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:50.978679  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 105517us
W20260709 13:00:50.986902  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81381.1us
I20260709 13:00:50.977463 11603 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:50.991331  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:50.939093 11644 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:50.995349 11768 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:50.995666 11656 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:50.996683 11759 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.314s	user 0.000s	sys 0.000s
W20260709 13:00:50.998250 11753 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:00:50.970510 11541 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:50.985373 11683 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:50.998281  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 430) took 2535 ms. Trace:
I20260709 13:00:50.985030 11703 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:51.004362 11768 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.011725 11768 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.008517  8748 rpcz_store.cc:276] 0709 13:00:48.462653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.842390 (+2379737us) service_pool.cc:224] Handling call
0709 13:00:50.842902 (+   512us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:50.873554 (+ 30652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.873600 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:50.874272 (+   672us) log.cc:844] Serialized 52 byte log entry
0709 13:00:50.962351 (+ 88079us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.962387 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.962455 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.997877 (+ 35422us) raft_consensus.cc:1685] finished
0709 13:00:50.997904 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.998066 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29602,"raft.queue_time_us":34050,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6125,"thread_start_us":87650,"threads_started":1,"wal-append.queue_time_us":121325}
I20260709 13:00:51.020334  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.020957  8816 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.014513 11703 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.023468  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:51.025050  8787 connection.cc:256] RPC call timeout handler was delayed by -2.4973626739999872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:00:51.025528  8787 connection.cc:256] RPC call timeout handler was delayed by -2.4926957870000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2669s for a response
W20260709 13:00:51.025832  8787 connection.cc:256] RPC call timeout handler was delayed by -2.4346164140000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 13:00:51.029760  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.030383  8818 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.030756  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:51.033228  8787 connection.cc:256] RPC call timeout handler was delayed by -2.2705626740000184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 13:00:51.033648  8787 connection.cc:256] RPC call timeout handler was delayed by -2.226462674000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
W20260709 13:00:51.033941  8787 connection.cc:256] RPC call timeout handler was delayed by -2.2183626739999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:00:51.034227  8787 connection.cc:256] RPC call timeout handler was delayed by -2.209573717000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:00:51.034518  8787 connection.cc:256] RPC call timeout handler was delayed by -1.7588719990000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
W20260709 13:00:51.034798  8787 connection.cc:256] RPC call timeout handler was delayed by -1.6747626740000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.284s for a response
W20260709 13:00:51.030792 11688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.400s	user 0.023s	sys 0.026s
W20260709 13:00:51.041976  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1452) took 2811 ms (client timeout 1665 ms). Trace:
W20260709 13:00:51.042506  8743 rpcz_store.cc:269] 0709 13:00:48.203420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.319620 (+116200us) service_pool.cc:224] Handling call
0709 13:00:51.014150 (+2694530us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540013bad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:51.014381 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413286144}
I20260709 13:00:51.047070  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.047782  8823 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.048321 11390 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:51.049710 11541 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:51.050854  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.050904 11541 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.006851 11603 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:51.044224 11688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.502s	user 0.023s	sys 0.027s
W20260709 13:00:51.081815  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45205.2us
W20260709 13:00:51.064926  8740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.007s	sys 0.005s
I20260709 13:00:51.065760 11718 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.066915 11683 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.067201  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:51.073657  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67489.9us
I20260709 13:00:51.074553 11649 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.075611 11603 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.083941  8818 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:00:51.090718  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.093519 11718 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:51.127986  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.096016 11644 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:51.096457  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.126085 11768 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:51.136205  8817 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0192ad465a724ab581b9529345f73bba"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:51.137256  8817 raft_consensus.cc:606] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:51.098814 11703 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:51.137732  8817 raft_consensus.cc:614] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:51.139367 11703 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:51.140213 11718 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:51.141615  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.127365 11629 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.169353  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 102300us
I20260709 13:00:51.170122  8816 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:51.133011 11485 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:51.184566  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 301501us
I20260709 13:00:51.171061 11649 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:00:51.189586 11649 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.190160 11649 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:51.191447 11649 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.193795  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.194804  8748 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.196467 11670 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.196620 11683 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:51.196980 11670 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:51.197531  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:51.197793  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.172911 11718 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:00:51.198498  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.181488  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:51.199936 11658 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.200847  8817 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:51.202060  8742 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.167789 11349 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.183089  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.199853  8741 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:51.210762  8817 raft_consensus.cc:606] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:51.209580  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:51.211391  8817 raft_consensus.cc:614] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:51.209175  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 82523.5us
I20260709 13:00:51.219017  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.220432  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.220222 11390 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:51.222111  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.222930  8743 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:51.223315  8744 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.223747  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.224856  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.186167 11624 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:51.229902 11624 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:51.230041 11537 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:51.230674 11537 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:51.231191 11624 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.231804 11537 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:51.233501  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1457) took 3001 ms. Trace:
I20260709 13:00:51.233846  8742 rpcz_store.cc:276] 0709 13:00:48.212610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.201555 (+2988945us) service_pool.cc:224] Handling call
0709 13:00:51.201894 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.210837 (+  8943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.213676 (+  2839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:51.222746  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1454) took 3018 ms. Trace:
I20260709 13:00:51.240659  8744 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:00:51.207562 11649 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.237982  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.246138  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.247682  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.248428  8816 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.173431 11629 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.254032  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.247704  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.178100 11605 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.258021  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.258330 11542 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.258870 11721 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 2.727s	user 0.006s	sys 0.018s
I20260709 13:00:51.260005 11658 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.264602  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.265035 11718 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.265456  8823 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:51.212445 11728 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:00:51.266498  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 92126.1us
I20260709 13:00:51.266971  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:51.267362  9412 connection.cc:256] RPC call timeout handler was delayed by -2.5018006040000387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:51.290038  9412 connection.cc:256] RPC call timeout handler was delayed by -2.2822976239999662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 13:00:51.290407  9412 connection.cc:256] RPC call timeout handler was delayed by -2.1283998000000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:00:51.290679  9412 connection.cc:256] RPC call timeout handler was delayed by -2.0833006040000441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
W20260709 13:00:51.290951  9412 connection.cc:256] RPC call timeout handler was delayed by -1.5578613160000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32000000000000006s for a response
W20260709 13:00:51.291276  9412 connection.cc:256] RPC call timeout handler was delayed by -1.3347646240000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 13:00:51.256596 11290 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:51.267884  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.268141 11768 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.266525  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.239032  8745 rpcz_store.cc:276] 0709 13:00:48.203920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.448611 (+244691us) service_pool.cc:224] Handling call
0709 13:00:48.448859 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.449036 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:48.452378 (+  3342us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.904043 (+2451665us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b18004461a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:00:51.095222 (+191179us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18004461a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:00:51.221993 (+126771us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.222222 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.222311 (+    89us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.222360 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.222498 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.222670 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19788,"prepare.queue_time_us":29096,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":42937,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":263,"spinlock_wait_cycles":70092416,"thread_start_us":2254327,"threads_started":2,"wal-append.queue_time_us":2185383}
I20260709 13:00:51.271229  8741 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:51.278250  8816 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:51.296168  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:51.280256 11791 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:00:51.281607  8817 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44b3244bf12f42aba79421575a96c621"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:51.298189  8891 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.298727  8898 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:00:51.299326  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.300026  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.293309  8821 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:51.288017 11531 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:51.297415  8889 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:00:51.299325  8712 connection.cc:256] RPC call timeout handler was delayed by -1.8109265730000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3148s for a response
I20260709 13:00:51.297811  8817 raft_consensus.cc:606] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:51.304255  8712 connection.cc:256] RPC call timeout handler was delayed by -1.7704265730000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31930000000000003s for a response
I20260709 13:00:51.286895  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.285519 11349 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:51.305157  8744 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:51.310091  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:51.307937  8817 raft_consensus.cc:614] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:51.292393 11688 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.333571  8748 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:51.270197 11595 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:00:51.307477  8893 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:51.332362  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.335319  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.303516 11531 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:51.336768  8894 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
W20260709 13:00:51.339725  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 450) took 4417 ms (client timeout 3200 ms). Trace:
W20260709 13:00:51.348645  8712 connection.cc:256] RPC call timeout handler was delayed by -1.27467656999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:00:51.349316  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2214051509999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:00:51.306180 11542 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:51.349704  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1854051509999408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 13:00:51.352164  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.335338  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.352514 11542 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:51.308210  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.353829  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.354210  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.354485 11605 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:51.355419  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:51.356691  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.358124 11688 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.344671  8822 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 4.
I20260709 13:00:51.384559  8743 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:51.366080  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.385659  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:51.348443  8820 rpcz_store.cc:269] 0709 13:00:46.919914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.921109 (+  1195us) service_pool.cc:224] Handling call
0709 13:00:51.303866 (+4382757us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400075b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:51.337460 (+ 33594us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3892870528}
I20260709 13:00:51.368192  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.369788  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.370316 11449 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:51.346414 11658 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:51.333313 11718 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:51.453125 11718 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.428975  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.429179  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.429572  8890 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:51.431813  8815 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:51.433334  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.432842  9449 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:51.398277  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.432583  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.457331  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.457568  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.458246  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:51.458470  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.459173  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.459751  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.382421 11531 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.438097  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.371153 11541 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:00:51.441089  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.370535 11290 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.442691  8896 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.436497  9445 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:51.445441  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.446431  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.466822  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.447427  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.447659  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.467705  8891 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:51.448354  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.450479  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.470114  8819 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:51.451144 11704 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:51.459973  8898 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:51.436039  8824 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:00:51.477119  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.463378  8889 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.467485  9443 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:51.443982  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.465248  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.456174  8817 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:51.473752  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.473866  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.477516  8893 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:51.479384  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.481599  8822 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:51.482312  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.502712  9440 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.383817 11670 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.504062  8820 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:51.504143  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.504981  9443 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:51.507907  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.508713  8895 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.509524  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.509716  9441 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:51.510136 11670 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.510576  9444 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.484149 11688 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:51.486822  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.484171  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.457341  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.500198  8823 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 4.
I20260709 13:00:51.486514 11290 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:51.462311 11728 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.503878  9449 raft_consensus.cc:2376] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7. Re-sending same reply.
I20260709 13:00:51.483693  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1456) took 3279 ms. Trace:
I20260709 13:00:51.510303  8895 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.444155 11449 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:51.501330  9442 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:51.523244  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.524308  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.531085  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.532194  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.532438  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.532881  8895 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:51.533144  8889 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:51.533998  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.534493  8889 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:00:51.535218  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.535825  8889 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.535714  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:51.536515  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.536828  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:51.537603  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.538355  9441 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:51.539079  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.539332  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.532495  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.552542  8815 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:51.555002  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.552767  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.555123  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:51.559916  8821 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:51.560030  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.561002  8896 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.561964  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:51.562239 11801 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:00:51.562690  8896 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.563580  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.564178  8896 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:51.518303 11531 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.565536 11794 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:51.565647  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.566500  8894 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.566277 11794 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:51.567299  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.567941  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.568488  8892 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.568783  8890 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.568848 11794 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:51.552220 11519 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.569804  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.541807  8819 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:51.554723  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.572893  8898 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:51.571101  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:51.565120  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.502022 11658 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:51.579129  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.582826  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:51.573774  8896 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:51.567381  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.575716  9447 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:51.519062  8747 rpcz_store.cc:276] 0709 13:00:48.204427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.758424 (+2553997us) service_pool.cc:224] Handling call
0709 13:00:50.806189 (+ 47765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.806764 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:51.165952 (+359188us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.483363 (+317411us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.483412 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.483451 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.483509 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.483601 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29368,"prepare.queue_time_us":359350,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2382,"raft.queue_time_us":46804,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":622731,"threads_started":2}
I20260709 13:00:51.586552  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.588577  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:51.590967  8819 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:51.593698  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:51.595199  8889 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:51.593943  8748 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.595227  8896 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:51.599108  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:51.600636  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63893.6us
I20260709 13:00:51.604084  8893 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:51.589481  8892 raft_consensus.cc:2376] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5. Re-sending same reply.
I20260709 13:00:51.516372 11704 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:51.605860  9445 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:51.606024  8894 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:51.601101  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.515969 11603 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:00:51.599727  8891 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:51.606859  8815 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:51.594682  8712 leader_election.cc:336] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.562s (SENT)
I20260709 13:00:51.607795  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:51.605705 11290 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.614430  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:51.610888 11290 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.618541 11290 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.618726  8817 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:51.621541 11649 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:51.621960 11649 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:51.622305 11649 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.623100  8889 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:51.618463  8890 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:51.626832  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:51.627426  9445 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.629905 11704 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.634819  8748 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:51.594406 11669 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.554625 11728 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:51.580039 11349 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:00:51.584052 11768 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:00:51.651906  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:51.667357  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53061.4us
I20260709 13:00:51.685549  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:52.249512  8821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.014s	sys 0.015s
W20260709 13:00:52.250017  8821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.014s	sys 0.015s
I20260709 13:00:52.250162  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1455) took 4046 ms. Trace:
I20260709 13:00:52.250218 11683 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:51.704845  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.253901 11683 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:52.254446  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.612462 11794 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.251170  8815 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:52.255061  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.254863 11683 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:52.255538  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.656559  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:51.836645  8749 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:00:51.893671  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.003664  9445 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:52.254276  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.775835  8890 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:00:51.788327  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 206103us
I20260709 13:00:51.930140  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:51.748620  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:51.658567  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.262521  9441 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:51.645613  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44004.5us
W20260709 13:00:52.257843  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.256636  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.274220  8822 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:52.044006  9444 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
I20260709 13:00:52.274793  8824 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:51.651506 11805 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:00:51.787750 11800 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:00:51.793447 11253 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:52.277186  8816 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.644912 11541 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.714809 11542 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:51.682122 11595 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.281004  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:52.136209 11786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.066s	sys 0.067s
I20260709 13:00:52.114688 11605 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.283105  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.283758 11542 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:52.256860  8740 rpcz_store.cc:276] 0709 13:00:48.204195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.467749 (+263554us) service_pool.cc:224] Handling call
0709 13:00:48.468084 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.613705 (+145621us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:48.614017 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.995853 (+2381836us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:51.099033 (+103180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.127665 (+ 28632us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:51.127729 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.249865 (+1122136us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54004646d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:52.249922 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.250091 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":550,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":11329,"raft.queue_time_us":209938,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":211,"spinlock_wait_cycles":353196160,"thread_start_us":2466988,"threads_started":2,"wal-append.queue_time_us":2054705,"wal-append.run_cpu_time_us":2931,"wal-append.run_wall_time_us":1338694}
I20260709 13:00:52.284417  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:52.284715 11688 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:52.285503 11349 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.285854 11649 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.663s	user 0.012s	sys 0.000s
I20260709 13:00:52.288129  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.288484  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.299943  8823 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.288810  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.288915  8890 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.289511  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.289616  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.302635  8897 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:52.290347  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.656548 11290 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.291395  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:51.862092 11449 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.293424  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.296581  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:52.280228  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.283959  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.291039  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:52.294378 11786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.069s	sys 0.069s
I20260709 13:00:52.304020  8816 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:52.304867  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:52.312465  9445 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.310017  8891 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:52.313721 11449 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:52.314450 11519 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.000s
I20260709 13:00:52.314778  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.315428  8891 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.317821  8748 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.318670 11592 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:52.319211 11592 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.321228 11592 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:52.314889 11519 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.305081  8895 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.299245 11542 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.306051  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.326046  8824 raft_consensus.cc:2410] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:00:52.325977  8748 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:52.310214  9441 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:00:52.299546 11688 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:52.329742  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:51.829599 11658 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.330009  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.333909  8894 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:52.304561 11605 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:52.332001  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:52.332288 11290 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:52.330606  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.295538 11541 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.335179  8891 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:52.339807  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.333580  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:52.335188  8747 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 4.
I20260709 13:00:52.336494  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.336937  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.297997 11595 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.335636  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:52.340843 11670 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.000s
I20260709 13:00:52.342100  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.344089  8743 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:52.345952  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.304970 11649 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.346724  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.349592  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:52.350060  8787 leader_election.cc:336] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.578s (SENT)
I20260709 13:00:52.350143  8894 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:52.349023  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:52.347901  8891 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.350864  9445 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:52.349046  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.351912  8895 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:52.352918  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.353274  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.355005  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.357414  9440 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 6 LEARNER]: Advancing to term 8
I20260709 13:00:52.358161  9443 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.358364  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.359107  8895 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.355549 11649 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:52.356024 11669 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:52.363057 11669 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.363597 11669 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.363094 11688 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:52.364717 11688 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.356201  9446 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:52.366674 11595 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:52.360078  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:52.350579  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.353926  8896 raft_consensus.cc:2376] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8. Re-sending same reply.
I20260709 13:00:52.329604 11349 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.371053  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.370661  9441 raft_consensus.cc:2376] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:00:52.406082 11683 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.371992  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.406945  8824 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:52.371502  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 109316us
I20260709 13:00:52.408193 11349 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:52.408999  8712 leader_election.cc:336] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.669s (SENT)
I20260709 13:00:52.409230  9440 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:52.409498  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:52.389684 11812 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:00:52.347368 11704 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.390421  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.420569  8822 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:52.420745  8894 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:52.392110 11656 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:52.393441  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.397199  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:52.398070  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.421463  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.421864  8891 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.399137 11773 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:52.399225  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55294.1us
I20260709 13:00:52.400641 11605 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:52.404058  8898 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:52.369659  8820 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:52.406601  8895 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.370289  9444 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:52.410987  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:52.432042  8787 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.587s (SENT)
I20260709 13:00:52.432366  8896 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:52.432514  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.413640  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:52.390673 11813 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:00:52.421518  8893 raft_consensus.cc:2376] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8. Re-sending same reply.
I20260709 13:00:52.434229  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.431367  8824 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:52.481906  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.423532 11704 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:52.482563  8824 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.482034  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.433501 11390 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.484572  8822 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.485543  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.486131  8893 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.397415 11669 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.494634  8824 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:52.495703  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.496325  8823 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:52.487385 11704 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.498961  8823 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
W20260709 13:00:52.500123  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91182.7us
W20260709 13:00:52.500713  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75550.5us
I20260709 13:00:52.504313  8822 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:52.505177  8822 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:52.506038  8824 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.500181  8818 pending_rounds.cc:85] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 2
I20260709 13:00:52.508670  8822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:52.509830  8823 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:52.507985  8818 pending_rounds.cc:107] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:52.512212  8818 raft_consensus.cc:2889] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.509186  8822 raft_consensus.cc:606] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.519690  9442 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:52.519690  8815 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:52.519747  8818 pending_rounds.cc:107] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 13:00:52.521342  8712 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.741s (SENT)
I20260709 13:00:52.521466  8818 raft_consensus.cc:2889] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.523658  8824 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:52.525561  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.526326  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.522609 11773 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.527527  8740 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:52.527704  8747 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:52.528298 11669 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:52.528586 11815 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:00:52.536000 11605 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 13:00:52.537144 11810 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:00:52.542810  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.543480  8742 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:52.543756  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 441) took 1361 ms. Trace:
W20260709 13:00:52.542039  9414 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.890s (SENT)
I20260709 13:00:52.545111  8746 rpcz_store.cc:276] 0709 13:00:51.182199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.197368 (+ 15169us) service_pool.cc:224] Handling call
0709 13:00:52.543582 (+1346214us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b5400464bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:52.543704 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219941248}
I20260709 13:00:52.552846 11768 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.553462 11768 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.554047 11670 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.554741 11670 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.556113 11768 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:52.558073  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.559785  8713 leader_election.cc:336] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.536s (SENT)
W20260709 13:00:52.560585  8713 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.563s (SENT)
I20260709 13:00:52.563212  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1433) took 1004 ms. Trace:
I20260709 13:00:52.527041 11817 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:00:52.541405 11390 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:52.568974  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 74294.9us
I20260709 13:00:52.569780  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.571000  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.573838  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.579691  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:00:52.583308  8712 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.935s (SENT)
I20260709 13:00:52.583997  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.585738  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.585952 11595 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:52.586375  8746 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.587530 11519 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:52.588061 11519 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:52.577983 11669 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:52.589775 11669 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:52.590749 11669 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.591382 11817 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.592633 11817 consensus_queue.cc:1243] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:52.585777 11656 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.594774 11656 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:52.595142 11656 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.602051  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.603057  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:52.603564  9414 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.967s (SENT)
I20260709 13:00:52.604403  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.604699  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.605631  8740 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:52.605950  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2078) took 1030 ms. Trace:
I20260709 13:00:52.606594  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.606359  8817 rpcz_store.cc:276] 0709 13:00:51.575594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.619475 (+ 43881us) service_pool.cc:224] Handling call
0709 13:00:51.619769 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.619971 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.619995 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.620448 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:00:51.944877 (+324429us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:51.944915 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.944996 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.605707 (+660711us) raft_consensus.cc:1685] finished
0709 13:00:52.605724 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.605888 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":324059,"threads_started":1,"wal-append.queue_time_us":727274}
I20260709 13:00:52.607532  8743 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.608043 11773 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:52.608654  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.602458  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:52.611780  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 217171us
I20260709 13:00:52.613065  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.614471  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.574518 11670 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:52.608470 11773 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:52.578655 11592 raft_consensus.cc:2764] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:52.618889  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.619813  8746 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:52.620950  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.622005  8744 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.622514  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.588994 11519 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.623520  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.624010  8898 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.626938 11656 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.563733  8821 rpcz_store.cc:276] 0709 13:00:51.556024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.561200 (+  5176us) service_pool.cc:224] Handling call
0709 13:00:51.581935 (+ 20735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.625465 (+ 43530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.625498 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.626055 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.250428 (+624373us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:52.541388 (+290960us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b18003eb600. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf
0709 13:00:52.541571 (+   183us) log.cc:844] Serialized 14 byte log entry
0709 13:00:52.549728 (+  8157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.549835 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.549893 (+    58us) raft_consensus.cc:1685] finished
0709 13:00:52.557943 (+  8050us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.558125 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8142,"prepare.queue_time_us":81919,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":43874,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":83134,"spinlock_wait_cycles":20322176,"thread_start_us":635021,"threads_started":2,"wal-append.queue_time_us":634214}
I20260709 13:00:52.630080 11656 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:52.641443  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:52.587567 11595 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:00:52.644834 11542 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:52.645601  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69953.3us
W20260709 13:00:52.646590  8712 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.941s (SENT)
I20260709 13:00:52.647142  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.648324 11421 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.616415 11649 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:00:52.648953  8712 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.962s (SENT)
I20260709 13:00:52.649643  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.667872  8898 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:52.665020  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.615849 11821 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.663275  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.686897  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.651885  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:52.680517  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65865.6us
I20260709 13:00:52.653679  8893 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.656488  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.612020 11531 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.696808  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.661437  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2075) took 1135 ms. Trace:
I20260709 13:00:52.697836  8816 rpcz_store.cc:276] 0709 13:00:51.526486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.532160 (+  5674us) service_pool.cc:224] Handling call
0709 13:00:52.661199 (+1129039us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b540024a4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:52.661357 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624041856}
W20260709 13:00:52.698388  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57036.3us
I20260709 13:00:52.701184  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.705366  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:52.705734  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 157791us
I20260709 13:00:52.649045 11542 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:52.656272 11595 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:52.709203  8898 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 5: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.709059  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.654070 11728 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:52.734604  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 114270us
I20260709 13:00:52.697175 11821 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:52.742061  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:52.743836  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.739959  8893 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:00:52.655783 11629 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:52.736837  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:52.744427  8713 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.852s (SENT)
I20260709 13:00:52.736171  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.735390  8820 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:52.744951  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.745066  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.746080  8893 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:52.746439  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.700323 11390 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.747840  8745 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:52.747407  8890 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.747928  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:52.752104  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.752727  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 165988us
I20260709 13:00:52.754223  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:52.756448  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 444) took 1567 ms (client timeout 1890 ms). Trace:
I20260709 13:00:52.756467  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:52.756891  8749 rpcz_store.cc:269] 0709 13:00:51.189053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.208605 (+ 19552us) service_pool.cc:224] Handling call
0709 13:00:52.756302 (+1547697us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400463cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:52.756393 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536632576}
I20260709 13:00:52.757262  8893 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.758249  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:52.703400 11421 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:52.732730 11683 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.732996 11624 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.733297 11595 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:52.742607 11706 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:52.769825  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.768707  8893 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:52.775542  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 591) took 1321 ms. Trace:
I20260709 13:00:52.763142  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.780550  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.780890  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.763609  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.780161  9448 rpcz_store.cc:276] 0709 13:00:51.440657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.443406 (+  2749us) service_pool.cc:224] Handling call
0709 13:00:52.760208 (+1316802us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:52.762295 (+  2087us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472785280}
W20260709 13:00:52.773905  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 125099us
W20260709 13:00:52.766899  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.654313 11773 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.775224  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.782151  8893 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.767294  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.782227  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.783638  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.784304  8889 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.786206 11773 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.786374  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.786811  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.787755  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.788069  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.788380  8894 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:52.788987  8897 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.747596 11821 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.786823 11596 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:52.789857 11596 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.790247 11596 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.790310 11728 raft_consensus.cc:2764] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:00:52.792670  8744 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:52.792732  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:00:52.793697  8819 raft_consensus.cc:606] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.793820  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.794075  8819 raft_consensus.cc:614] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:52.794790  8741 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:52.795544  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:52.762284 11605 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:52.803633 11605 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:52.804773 11605 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.805464 11656 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:52.805888 11605 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.805989 11656 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:52.807193 11656 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.803073 11704 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:52.798034  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.809448  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.810509  8896 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:52.810521 11718 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.811189 11718 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.811758  8896 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:52.812640 11349 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:52.827849  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.781971  8895 raft_consensus.cc:2376] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8. Re-sending same reply.
I20260709 13:00:52.781385 11683 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:52.803045  8712 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.018s (SENT)
I20260709 13:00:52.845064  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.829847  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.780143 11595 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.846714 11819 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.846865  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:00:52.813211  8896 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:00:52.817144  8891 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:52.828269  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.850266  8745 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.851029  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.815091 11449 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:52.851606 11449 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:52.852085  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.852406 11449 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:52.846705 11683 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:52.836768  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.841573  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1435) took 1260 ms. Trace:
I20260709 13:00:52.808564 11656 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.832840  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.826014  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:52.821152 11649 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:00:52.853238  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.827342  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.860473  8741 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.818509 11541 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:52.853654 11449 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.813624 11349 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:52.796932  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.837170  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.866469  8898 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.824586  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:52.814978 11718 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:52.797017  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.865746  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 67896us
I20260709 13:00:52.870417  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:52.871004  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.871465  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.871922  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.854390  8746 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.853793  8743 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:52.858829  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.856105  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:52.873401  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48903.7us
I20260709 13:00:52.859323  8893 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:00:52.874178  8893 pending_rounds.cc:85] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 6
I20260709 13:00:52.762144 11421 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.860620  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.872421  8741 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:52.874030  8742 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:52.876349  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.816030  8894 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:52.831254  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.877867  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:52.874629  8893 pending_rounds.cc:107] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:00:52.878690  8893 raft_consensus.cc:2889] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.879156  8893 pending_rounds.cc:107] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:00:52.879528  8893 raft_consensus.cc:2889] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.881335  8824 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.882383  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.853682  8818 rpcz_store.cc:276] 0709 13:00:51.581145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.494946 (+913801us) service_pool.cc:224] Handling call
0709 13:00:52.500025 (+  5079us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:00:52.550699 (+ 50674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.550748 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:00:52.561940 (+ 11192us) log.cc:844] Serialized 78 byte log entry
0709 13:00:52.716165 (+154225us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:52.760655 (+ 44490us) log.cc:844] Serialized 14 byte log entry
0709 13:00:52.761321 (+   666us) log.cc:844] Serialized 14 byte log entry
0709 13:00:52.820416 (+ 59095us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b18002590c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:00:52.840691 (+ 20275us) log.cc:844] Serialized 14 byte log entry
0709 13:00:52.840943 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.841231 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.841277 (+    46us) raft_consensus.cc:1685] finished
0709 13:00:52.841318 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.841502 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52493,"prepare.queue_time_us":75247,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":107,"raft.queue_time_us":94497,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":12274,"spinlock_wait_cycles":47161728,"thread_start_us":105592,"threads_started":1,"wal-append.queue_time_us":176212}
I20260709 13:00:52.884302  8892 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:00:52.875509  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.886644  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.887844  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:52.888552  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 93095.6us
I20260709 13:00:52.890416  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.890736  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.895537  8822 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:52.896636  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.896797  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.897225  8898 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:52.897568  8898 pending_rounds.cc:85] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 4
I20260709 13:00:52.897781  8898 pending_rounds.cc:107] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:52.897964  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.897992  8898 raft_consensus.cc:2889] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.900146  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.900833  8747 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:52.901212  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.901298  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.902455  8891 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:52.833312 11704 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:52.900369  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.907189  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:52.907701  8821 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:52.907796  8713 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.016s (SENT)
I20260709 13:00:52.903959  8823 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:52.909754  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.909981  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.873997 11821 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.910328  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.905565  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.902941 11718 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:52.910013  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.918711  8823 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:52.865562 11349 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:52.921577  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.922317  8893 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.857504 11706 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:52.911109  8822 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:52.905383 11688 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:52.873709 11658 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.917058  8896 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:52.923796  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.894690 11773 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:52.908289  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:52.918057  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.917590  8748 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.926167  8822 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.926479  8891 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:52.919801  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:52.892602 11683 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:52.903468 11519 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.923038  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:52.932430  8893 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:52.919078  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.934453  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.936389  8820 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.936663  8743 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:52.927599  8892 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:52.885622 11421 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:52.928545  8817 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:52.929185  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.875267 11542 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:00:52.906879 11290 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.937971  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.941951  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:52.939121  8748 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:52.939214  8741 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:52.943045  8822 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:52.940806  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:52.944542  8746 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:52.946357  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.946990  8745 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:52.953591  8744 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6. Re-sending same reply.
W20260709 13:00:52.961627  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 70297.1us
I20260709 13:00:52.962934  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:52.964990  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57258us
W20260709 13:00:52.965816  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.966054  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:52.967190  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.965003  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.968572  8742 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:52.941340 11718 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:52.971342  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:52.971909  8712 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.104s (SENT)
I20260709 13:00:52.947722 11773 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:52.972956  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.949234 11683 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:52.975417  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:52.928465 11658 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:52.968384 11817 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.968248 11541 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:52.979918  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:52.992756  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:52.973469 11290 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.992973 11826 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:52.999753 11826 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:00:52.999989 11832 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:00:53.000253 11826 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.988946 11794 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.999845  8747 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:53.008591  8821 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:53.010136  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:53.010390  8747 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.969573 11596 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.014374  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.013540 11817 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.996933 11669 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.020154  8747 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.017894 11817 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:53.018283 11592 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.023181 11658 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:53.039109 11658 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.038921 11595 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.023452 11817 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.024175 11718 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.041253 11595 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:53.029704  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:53.021471 11838 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.046510  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.029996  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:53.033430  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.031514 11669 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.047739  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:53.041513  8862 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.167s (SENT) [suppressed 93 similar messages]
I20260709 13:00:53.048573 11629 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.000870 11683 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.033838  8815 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:53.049743 11817 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.050392  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.051884 11817 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:53.023883  8712 leader_election.cc:336] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.121s (SENT)
I20260709 13:00:53.052603  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.011801 11349 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.047338  8742 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:00:53.037103  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 94414.8us
I20260709 13:00:53.046177 11773 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.056453  8895 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:53.058102  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.059216  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:53.074880  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 139433us
W20260709 13:00:53.090519  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.093040  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:53.093771  8816 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:53.047317 11595 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.027402 11596 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:53.067129 11629 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.098272  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.074646 11644 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:53.099212  8821 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:53.091723 11531 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.101681  8815 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.092280 11768 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:53.102411  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78586.9us
I20260709 13:00:52.997473 11706 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:53.109229  8863 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.665s (SENT)
I20260709 13:00:53.100332 11644 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.110914  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.109719  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.113796  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.115986  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:53.117601  8789 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.880s (SENT)
I20260709 13:00:53.118044  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.103112 11768 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.115413 11821 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:53.135922 11449 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.136718 11449 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.125131 11826 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.139282 11826 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:53.103674 11704 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:53.149538  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75961.8us
I20260709 13:00:53.116716 11644 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.112315 11670 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.155436  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.156808  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:53.157748  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48598us
I20260709 13:00:53.141434 11768 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.134366 11794 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.134198 11819 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:53.164875  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58176.8us
W20260709 13:00:53.166478  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 90987.1us
I20260709 13:00:53.168193  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.169010 11852 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:00:53.173007 11835 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:00:53.165727 11541 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.174197 11519 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.177311 11841 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.178145 11841 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:53.178592 11841 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:53.179450 11841 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.180258 11421 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.160674 11844 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:00:53.122588 11531 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:00:53.183147  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:53.164386 11819 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.190173  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.190376 11819 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.157859 11670 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:53.198406  8787 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.204802 11644 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:53.206388  9414 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.932s (SENT)
I20260709 13:00:53.206851  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.207973 11656 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.209534 11649 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:53.209719 11670 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.210958 11670 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.205504 11840 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.213542 11840 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.213936 11840 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.197744 11794 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:53.217650 11794 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:53.217798 11592 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.217960  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.218644  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.218029 11794 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.218412 11592 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:53.221076 11821 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.221534 11821 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.221915 11821 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.221318 11592 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.223213 11768 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.228242 11624 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.228681 11624 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:53.230904 11656 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.231123 11838 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.229523 11624 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.232795 11624 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.235672  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.239928  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.240782  9445 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 7. Ops: []
W20260709 13:00:53.244117  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.244333  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.244740  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:53.244880  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.244949  8893 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.245759  8891 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.250560  9446 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:00:53.250985 11421 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:53.252192  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.253152  9444 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:53.265228  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.265949  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:53.262962  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.265249  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:53.266942  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:53.267611  8889 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:53.269322  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101254us
I20260709 13:00:53.269502  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.273826  9441 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:53.270974  9446 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.271230  8892 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:53.270777  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.262133  8893 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:53.274222  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:53.289495  9449 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.291715  9443 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:53.296056  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.296242  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.301520 11794 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.303473 11794 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:53.305958  8824 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:00:53.306541  9448 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:00:53.308648 11704 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:53.312170 11821 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.313273  8816 raft_consensus.cc:2368] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
W20260709 13:00:53.314916  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.322495  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.323452  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 133403us
W20260709 13:00:53.338191  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.253450 11596 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:00:53.255793 11605 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:00:53.353364  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.354521  9446 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:53.355707  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.356981  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.253654 11819 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.352583  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.366545  9441 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 8. Ops: []
I20260709 13:00:53.367848  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.370515  9443 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:53.352072 11519 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.377264  9445 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:53.380203 11688 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:53.380329  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.380645 11688 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:53.381330  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.381567 11688 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.382730  9442 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:53.383912  9440 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:53.384130  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.383183  8891 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:53.380226  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.389074  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.347527 11839 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:00:53.389948  9447 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:53.406055  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.406661  9449 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:00:53.404565  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.752s (SENT)
W20260709 13:00:53.411269  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43541.8us
I20260709 13:00:53.404763  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:00:53.416913  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.417554  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.422165  9441 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:53.426524  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.427196  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.427771  9449 raft_consensus.cc:2376] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:00:53.389210 11857 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:53.430588  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.463114  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:53.474407  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.450021  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.465703  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.492875  9449 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:53.463965  8815 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:53.492951  9442 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:53.450500  9445 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:53.462183  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.440784  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:00:53.490314  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.471583  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.497738  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.498287  8898 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.501843  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:53.502471  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.503177  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.503435  9446 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:53.504092  9440 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.503772  8891 raft_consensus.cc:2376] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6. Re-sending same reply.
I20260709 13:00:53.426698 11688 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.507871  8822 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:53.508358  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 95643.3us
I20260709 13:00:53.510097  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.510955  8891 raft_consensus.cc:2376] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:00:53.511862  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:53.514890  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.515075  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.515838  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.516319  8896 raft_consensus.cc:2376] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:00:53.517069  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:53.517804  8817 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:53.518527  8821 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:53.480661 11519 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.519625 11519 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:53.515794 11253 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:00:53.520586 11519 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.444097 11819 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:53.521399 11605 raft_consensus.cc:2764] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:53.521531  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:53.515478  8897 raft_consensus.cc:2376] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7. Re-sending same reply.
I20260709 13:00:53.529313  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:53.530849  8898 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:53.525633  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.536409  9443 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:53.538563  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.538899  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.539348  9442 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:00:53.540923  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78879.8us
I20260709 13:00:53.544273  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:53.549451 11519 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.552942  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.510371 11857 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:00:53.561071  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.544967 11858 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.562602  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.582048  9443 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 5: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:53.563405  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.575927 11531 raft_consensus.cc:2764] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:53.566627 11253 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:53.561640  8896 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:53.577625  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.578728  8890 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.453328 11840 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.580412 11773 raft_consensus.cc:993] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: : Instructing follower b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 13:00:53.580309  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.579921 11421 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:53.585824 11624 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:53.583150  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44350.2us
I20260709 13:00:53.592339  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.585981  8890 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.582509  9442 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:53.595805 11596 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.595930  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.596577 11596 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:53.597508 11595 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.597672  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.598486 11794 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:53.599073 11773 raft_consensus.cc:1081] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Signalling peer b60591217ee24378bb66a86d6d0dda52 to start an election
I20260709 13:00:53.600091  8890 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:53.602000 11390 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.603191  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.603729  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.607620  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.605890 11840 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:53.614125  8896 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:00:53.616918  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:53.622081  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.621999  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.626403 11449 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:53.627262 11449 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.627604 11449 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:53.627982 11449 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.628295  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.629034  9443 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:53.629189  9449 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:53.630004  9447 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.630131  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.630627  9449 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.632483 11390 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:53.632941 11390 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:53.593361 11768 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:53.633348 11839 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.633848 11839 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.635632 11706 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.636240 11854 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:53.636139 11706 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.636646 11854 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:53.638274 11706 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:53.639410  9445 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:53.643672  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.647408 11854 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.648887  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:53.649614  9442 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:53.650213  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:53.610628 11794 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:53.650801  8893 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.665723  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.659427  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:53.653818  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.674822  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.677958  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.681979  8898 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:53.685030  8893 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:53.648918 11531 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:00:53.691730  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61712.2us
I20260709 13:00:53.694057  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:53.695220  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.694795  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:53.698764  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:53.699191  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:53.700194  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:53.699384  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 77519.5us
I20260709 13:00:53.668465 11541 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:53.704052  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:53.699985  8894 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:00:53.677228 11595 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:53.648614 11449 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.691468 11253 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:53.722883  8824 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.718271  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.690829 11841 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.689553 11773 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:53.708518  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46112.3us
I20260709 13:00:53.734786  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.727613  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.735291  8819 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:53.733650  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 156155us
I20260709 13:00:53.706506 11768 raft_consensus.cc:2764] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:53.738292  8817 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:53.739024  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.701853 11605 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.696537 11531 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:53.707378  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:53.701678 11856 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.749826  8818 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.752033  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.723552 11854 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.718556 11706 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.756575  9449 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:00:53.719041 11718 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:00:53.717706 11390 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:53.772704 11595 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:53.734318 11820 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.733698 11253 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.736771 11858 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:53.783907  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.737066 11773 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:53.787043  8824 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:00:53.754880 11531 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.753113 11840 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:53.790117 11683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.011s	sys 0.036s
I20260709 13:00:53.806082 11605 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:53.793095 11253 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.775156 11871 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:53.806929  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 67986.4us
I20260709 13:00:53.813097  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:53.812918 11870 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:53.790889 11706 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:53.794209 11773 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.819593  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.802911 11541 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.816675  9443 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7. Re-sending same reply.
W20260709 13:00:53.810690 11683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.011s	sys 0.036s
I20260709 13:00:53.805119 11656 raft_consensus.cc:2764] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:00:53.842626  8818 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:53.805104 11872 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.744956 11794 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:53.804363 11881 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.809674 11820 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:53.850340  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 122827us
W20260709 13:00:53.854123 11867 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:00:53.854430 11869 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:53.857689 11595 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:53.859071 11656 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:00:53.814824 11821 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:53.868677  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:53.849834 11864 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:00:53.877142  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.811268 11768 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:00:53.884327  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 148058us
W20260709 13:00:53.885732  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 161515us
W20260709 13:00:53.887892  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 193249us
W20260709 13:00:53.917013  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 171299us
W20260709 13:00:53.884968  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 126642us
I20260709 13:00:53.926635  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.927508  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:00:53.930414  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:53.934893  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.935681  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:53.871326 11683 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:00:53.934309 11541 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.930598 11855 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:53.952014  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:53.955753  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.884929 11852 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:00:53.966219  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 224479us
W20260709 13:00:53.993942  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66383us
I20260709 13:00:53.932235 11872 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:00:53.994894  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:53.997630  9447 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 7 LEARNER]: Advancing to term 8
I20260709 13:00:53.938606 11840 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:53.995828  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 118793us
I20260709 13:00:53.948431 11880 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.994429  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:53.963505 11871 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:54.021582 11852 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.023036 11840 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:54.023900 11873 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:00:53.964569 11875 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:53.938261 11605 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.036151  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:54.083246  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 88930.4us
I20260709 13:00:54.084314  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:54.084391  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 250364us
I20260709 13:00:53.995855 11877 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.019161 11847 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:54.075083  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 63062.4us
I20260709 13:00:54.025053 11449 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:54.025267 11886 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:53.945606 11706 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.012889 11880 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:54.091203 11835 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:00:54.027254 11872 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:00:54.089951  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:54.111727  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:54.112357  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.098748 11856 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:54.116106  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 97249.2us
I20260709 13:00:54.117062  9412 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:54.066136 11876 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:00:54.123529  8818 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 13:00:54.096028 11838 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:54.124632  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:54.125303  8819 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.122370  9412 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:54.103469 11847 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.100699 11886 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.103215 11880 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.105330 11683 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.005905 11820 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.107729 11644 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:54.119578 11901 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:54.153847 11884 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:00:54.156356 11878 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:00:54.158560 11883 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:54.159333 11883 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.116024 11877 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:00:54.166963  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.168576 11847 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.169220 11644 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:00:54.169986 11885 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:00:54.160559 11883 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.139642 11871 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.165742 11875 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.138547 11902 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:00:54.214767  9447 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 8 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:54.170526 11898 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:54.205932 11900 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.205506 11899 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.215200  9447 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 4
I20260709 13:00:54.225903  9447 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:54.226374  8749 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.226554  9447 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 8 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.225610  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.228354  8742 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:00:54.224167  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122383us
I20260709 13:00:54.233773  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.234570  8747 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.218032 11644 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.235936 11882 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.236560 11826 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:54.236977 11826 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:54.237736  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.238405  8740 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:54.239490  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.241946 11670 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.242543 11670 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:54.218827 11349 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.245645 11875 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.248899 11844 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:54.222889 11904 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:54.237023 11882 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:54.246306 11688 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.000s
I20260709 13:00:54.250404  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.261760  8819 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:00:54.263235  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:54.280265  8815 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:00:54.286072  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 163835us
I20260709 13:00:54.289956 11838 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.291326 11838 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.292117 11541 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.293725 11883 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:54.245079 11670 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:54.296561  8889 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:00:54.296700  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.304427  8889 pending_rounds.cc:85] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 7
I20260709 13:00:54.304703  8889 pending_rounds.cc:107] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:00:54.304930  8889 raft_consensus.cc:2889] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.250346 11888 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:54.230938 11898 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.299777  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:54.300743  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.249465 11844 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.297479  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.298007  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.306120  8740 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:00:54.265475 11519 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.003s
I20260709 13:00:54.330780 11872 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:54.332106 11872 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:54.317641  8745 raft_consensus.cc:2376] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8. Re-sending same reply.
I20260709 13:00:54.335216  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.250064 11904 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.298348  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.309278  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.339986  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.340183  8744 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:54.341535  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:54.342293  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.307829  8819 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:54.311489  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.302266  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.287863 11840 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:00:54.321169  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:54.322058  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 370224us
I20260709 13:00:54.345072  8741 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:54.345775  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.345911  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.322104  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.346418  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:54.346395  8745 raft_consensus.cc:2376] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8. Re-sending same reply.
I20260709 13:00:54.347210  8746 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:54.346745  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:54.328895 11854 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.000s
I20260709 13:00:54.329392 11881 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.346530  8741 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:00:54.355384  9440 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:54.329633 11902 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:54.330240 11895 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:54.330190 11898 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.238045 11826 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.338796  8747 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:54.322491  8818 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:00:54.362030 11868 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:54.249229 11531 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.229274 11886 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:54.363246 11883 consensus_queue.cc:1175] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7fd110c312c14e17a2bbe6d453493cf2" responder_term: 8 status { last_received { term: 4 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: [7.8-7.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:54.354756  8743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:51668
I20260709 13:00:54.364387 11531 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.347935  8745 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:54.360143 11881 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.365768  8743 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:54.365957 11904 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.366149  8743 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.250451 11899 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.367031 11881 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.236578 11644 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.368259 11899 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:54.369630  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:54.350790  8744 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:00:54.354290  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:54.356740  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:54.310007 11888 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.411966 11888 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.413184 11888 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.410940 11868 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.415910 11868 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.418048 11899 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:54.356032  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 189161us
I20260709 13:00:54.421706  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.356079 11592 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.363768  9443 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:54.294162 11541 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.348364 11683 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.389731  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.390399  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.433435  8741 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.433763  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:54.433151  8744 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:54.436033  8744 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:00:54.397248  9449 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:00:54.405913  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.410138  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.410377 11889 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:54.438195 11768 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.410179 11881 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.438288  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:54.439148 11768 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:54.412577 11892 raft_consensus.cc:1064] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:00:54.439690 11768 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.441855  8741 raft_consensus.cc:2410] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:54.348064  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:54.359033  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:54.442523  8742 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.427284  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:54.442770  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 94807.3us
I20260709 13:00:54.443185  8749 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:54.423772 11253 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 13:00:54.425168  8743 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.360003 11895 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.438850 11881 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.410799  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:54.387146 11900 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.430481  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:54.443831  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.439273  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.423439  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.445685 11896 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.441256 11768 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.389580 11706 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:54.429347 11854 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:54.430104 11899 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:54.472201 11592 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:54.528944  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46345.3us
W20260709 13:00:54.537958  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 94236.3us
I20260709 13:00:54.432938 11683 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:54.559448  8863 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.554849  8742 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:54.432965 11870 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:00:54.563292  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.557735  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:54.566107  8749 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:00:54.489147 11670 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.575542  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.580044  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:54.581248 11916 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.582046 11907 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.582368 11910 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.582892 11910 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:54.585954 11909 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:00:54.586566 11845 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.006s	sys 0.013s
W20260709 13:00:54.586737 11913 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:00:54.586932 11845 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.006s	sys 0.013s
I20260709 13:00:54.588793 11905 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:54.589572 11905 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.590811 11905 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.592525 11670 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.567703 11688 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.561751 11683 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.562014 11872 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:54.482707 11854 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.551704 11706 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.607028  8820 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.528455 11840 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:00:54.456758 11253 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.626567  8817 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.628391  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:54.630609  8819 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
W20260709 13:00:54.632803  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52850.8us
I20260709 13:00:54.633752  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:54.603834 11773 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.759s	user 0.008s	sys 0.004s
I20260709 13:00:54.553622 11592 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.569473 11898 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 13:00:54.658512 11918 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:00:54.627964 11907 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.558733 11896 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:00:54.661023 11929 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.661528 11877 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:54.661998 11877 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.661613 11929 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.662890 11877 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.664018 11929 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:54.664393 11877 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.653033 11644 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.653275 11773 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:54.681162 11915 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:54.683576 11854 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:54.582835 11879 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:00:54.609019 11910 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.699326  8815 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:54.653743  9441 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.635654 11872 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:54.655889 11912 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:00:54.656582 11253 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.582039 11881 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.585072 11895 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.710335 11895 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:54.719952  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 144507us
I20260709 13:00:54.724561 11911 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:54.725164 11911 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.727315 11911 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.731137 11592 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.731933 11908 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:54.732669 11908 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:54.733772 11908 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:54.719564 11902 consensus_queue.cc:1175] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "29c7934ee528454785432732d01ed0a0" responder_term: 8 status { last_received { term: 7 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:54.745100 11706 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.749581 11872 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:54.750931 11844 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.753050 11844 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.755460 11519 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:54.755929 11519 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:54.756805 11519 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.758298 11519 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.759042  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:54.766988 11773 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:54.771168  8743 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:54.773914 11906 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.289s	user 0.000s	sys 0.002s
I20260709 13:00:54.776731 11886 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.782371 11624 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:54.782795 11624 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:54.783655 11624 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:54.784973 11624 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:54.788439 11892 raft_consensus.cc:1068] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Unable to promote non-voter 112105225cc140459688be5d31dcb071: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:00:54.790310 11596 consensus_queue.cc:1175] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:54.795193 11904 raft_consensus.cc:1064] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
W20260709 13:00:54.795680 11904 raft_consensus.cc:1068] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Unable to promote non-voter 112105225cc140459688be5d31dcb071: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:00:54.816190  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.817458  8890 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:54.819837  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.820801  8891 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:54.829258  8897 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:54.830400  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.831135  8894 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.847378  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.854311  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.854996  8896 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:54.860416  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:54.862846  8892 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:54.863355  8892 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 4
I20260709 13:00:54.863646  8892 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:54.863941  8892 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.832010 11876 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.826264 11644 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:54.877893  8896 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:54.874709 11595 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:54.879791  8816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.026s	sys 0.054s
W20260709 13:00:54.880270  8816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.026s	sys 0.054s
I20260709 13:00:54.891654  8891 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
W20260709 13:00:54.877373 11880 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
I20260709 13:00:54.903203  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.888305  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:54.923260  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.924079  8891 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 4.
I20260709 13:00:54.864650 11773 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:00:54.926155  8896 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:00:54.929246  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.930270  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:54.939730  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.901957 11870 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.919631 11595 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:54.927769 11821 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:54.957998  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.958856  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.959630  8896 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:00:54.009690  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:54.885891 11644 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.958914  8890 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:54.879928 11876 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.944967 11879 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.971608  8890 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:54.972962  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.980182  8892 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:54.974014  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.980886  8897 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.943482 11875 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.985123  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:54.958431 11821 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.925922 11880 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:54.947218 11904 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:00:54.987092 11875 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.071928  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:54.955143 11683 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:55.085691  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.004607 11821 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.089160  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:55.004226 11541 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:54.984333 11879 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.191854  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:55.257917  8743 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.164281 11839 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:55.166759  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:55.166371  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.179291  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.172104 11880 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.167953  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:55.166917 11933 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:00:55.165798 11935 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:55.159706  8820 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:55.190585  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.165093  8896 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:55.090440 11875 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:55.170641 11876 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.161609  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:55.286443  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:55.286580  8892 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
W20260709 13:00:55.160190  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:55.091225 11888 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.678s	user 0.004s	sys 0.008s
W20260709 13:00:55.167817 11934 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:00:55.223336  8819 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.165699  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:55.133714  8817 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.239517  8897 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:55.241029  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:54.955735 11870 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.303808 11870 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.271687  8894 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:55.239012 11768 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.247786 11896 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.286959  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.288537  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.369259  8747 raft_consensus.cc:2410] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:55.370412  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.371134  8747 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:55.372210  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.372844  8747 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:55.289783  8741 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:00:55.374599  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:55.375880  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.376686  8741 raft_consensus.cc:2368] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6.
I20260709 13:00:55.367997  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:55.383558  8747 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:55.133168 11595 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.278856 11839 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.305138 11897 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.305456 11624 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:55.305737 11819 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:55.306209 11936 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:00:55.308250 11826 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.309943 11644 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.310971  8896 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:00:55.311811  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.263861 11541 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.318676  8744 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:55.320061  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:55.321739 11932 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:00:55.322067  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 487) took 1821 ms. Trace:
I20260709 13:00:55.298813 11876 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:55.008793 11592 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.331730  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.279361 11920 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.334944  8819 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:55.458003  8740 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:55.337399 11605 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:00:55.363891  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1298616469999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:00:55.458634  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0317616469999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:00:55.459920  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:55.390286 11900 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:00:55.403707  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:55.293802 11935 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:55.368029 11519 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 13:00:55.415419  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.471411  8747 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:00:55.389544  9444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.016s	sys 0.024s
I20260709 13:00:55.360252 11768 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.413337 11839 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.457177  8741 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:55.412218 11847 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.463892 11920 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.479250 11847 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.480912 11920 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:55.482512  8823 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:55.473409 11819 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:55.483116  8817 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
W20260709 13:00:55.472913  9444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.016s	sys 0.024s
I20260709 13:00:55.470438 11706 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.460551 11876 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.296053  8743 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:55.389499 11896 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.512748  9445 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 10. (index mismatch)
I20260709 13:00:55.512943  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.467116  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:55.464071 11605 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.412149 11892 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:00:55.410833 11869 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:00:55.475376  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2091) took 1166 ms. Trace:
I20260709 13:00:55.492643 11706 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:55.478327 11940 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:00:55.478791 11541 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.510006  9443 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:55.462747 11592 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.481237 11847 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.523521  8816 rpcz_store.cc:276] 0709 13:00:54.308523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.309373 (+   850us) service_pool.cc:224] Handling call
0709 13:00:54.309649 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.309976 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:54.310003 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.334353 (+ 24350us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.880700 (+546347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:54.880755 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.880852 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.475172 (+594320us) raft_consensus.cc:1685] finished
0709 13:00:55.475191 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.475317 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8644,"raft.queue_time_us":7111,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":16091,"thread_start_us":509610,"threads_started":1,"wal-append.queue_time_us":600736}
I20260709 13:00:55.430351  8889 rpcz_store.cc:276] 0709 13:00:53.500984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.508459 (+  7475us) service_pool.cc:224] Handling call
0709 13:00:53.530807 (+ 22348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.305407 (+774600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.321825 (+1016418us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:55.321986 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700863744}
W20260709 13:00:55.526196  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59184.1us
I20260709 13:00:55.542884  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.527025  9441 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:55.534101 11931 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:55.534652 11896 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.525687 11881 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:55.536486 11839 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.515928 11519 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.463097 11683 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.548933  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.552505  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.553855 11871 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.554342 11519 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.556394 11519 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.553798 11881 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:55.524919 11879 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.540282 11706 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.538674 11605 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.558171 11794 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.564355 11947 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.569144 11881 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.569728  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2090) took 1261 ms. Trace:
I20260709 13:00:55.570240  8815 rpcz_store.cc:276] 0709 13:00:54.308255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.308928 (+   673us) service_pool.cc:224] Handling call
0709 13:00:54.309184 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.699905 (+390721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.569439 (+869534us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b540012be54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:55.569653 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897007360}
I20260709 13:00:55.547533 11950 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:55.577430  8748 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:55.579051 11768 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.580394 11935 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.581138  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:55.585441 11794 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:55.586724 11794 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:55.544900 11931 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.629530 11875 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:55.468528  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 64955us
I20260709 13:00:55.736523  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:55.405992 11870 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:55.589570 11519 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.599498 11920 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.601729  8816 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:00:55.602283 11839 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.602212 11869 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:55.603080 11704 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.611889  8817 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:55.612478  8820 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:55.611893  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:55.756147  8817 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:55.614567 11876 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:55.615489  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:55.618914 11935 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:00:55.623477 11895 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.913s	user 0.006s	sys 0.006s
I20260709 13:00:55.557233 11592 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:55.632493  8864 leader_election.cc:336] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.539s (SENT)
I20260709 13:00:55.637997 11768 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:55.624912 11931 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.758577 11704 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.653587  8898 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:55.654070 11844 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.001s
I20260709 13:00:55.654157 11857 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:55.656740  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.662078 11901 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:55.662602 11819 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:55.665699 11624 raft_consensus.cc:1064] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:00:55.671281  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:00:55.781805 11624 raft_consensus.cc:1068] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Unable to promote non-voter 112105225cc140459688be5d31dcb071: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:00:55.781718 11819 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:55.782063  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 110926us
I20260709 13:00:55.783630  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:55.783950 11819 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:55.680374  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.752461 11519 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.786092  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.787346 11519 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.787647  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.788436  9447 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:55.681082  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.681479  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.682003 11852 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:00:55.681008  9443 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:55.683485  9449 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:55.685679 11905 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.686256 11855 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.687747  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.692186  9445 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:00:55.692975  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:55.807780  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 114934us
I20260709 13:00:55.809422  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:55.693499 11903 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.696179 11883 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:55.813591  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.704001 11531 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:55.823251 11531 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.707947  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:55.824193  8818 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:55.824276 11931 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.813015 11844 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:55.711277 11882 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:55.694000  8822 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:55.825433  8822 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:55.715073  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1770) took 1418 ms (client timeout 1819 ms). Trace:
I20260709 13:00:55.560568  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.733845  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 608) took 1381 ms. Trace:
I20260709 13:00:55.587154 11794 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.826719  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:55.748313 11904 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:55.762493  8746 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:55.763219 11935 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.832427 11875 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.832681  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:55.833812  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:00:55.789227 11901 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.860080  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.764004 11768 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:55.754935 11944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.045s	sys 0.095s
I20260709 13:00:55.865084  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:00:55.866391  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 305998us
I20260709 13:00:55.812732 11624 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:55.643415 11877 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.979s	user 0.004s	sys 0.008s
I20260709 13:00:55.750347 11421 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:00:55.834244 11892 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.833868 11935 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.880128  8742 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:55.865824 11944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.045s	sys 0.095s
I20260709 13:00:55.861013 11875 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.821475 11957 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:00:55.821717 11869 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:55.822183  8749 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:55.822417 11868 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:55.822785  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.825008 11844 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:00:55.826102  8748 rpcz_store.cc:269] 0709 13:00:54.296293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.296439 (+   146us) service_pool.cc:224] Handling call
0709 13:00:55.714916 (+1418477us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:55.715019 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2928196992}
I20260709 13:00:55.413573 11897 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.754400 11920 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.756816 11839 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.805672 11907 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:55.831161 11704 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:55.777993 11857 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.829217 11903 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.763862  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:55.952898  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 320546us
W20260709 13:00:55.838721  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.009s	sys 0.071s
W20260709 13:00:55.954141  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.009s	sys 0.072s
W20260709 13:00:55.954473  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 777430us
W20260709 13:00:55.827976 11821 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.003s
I20260709 13:00:55.826406  9444 rpcz_store.cc:276] 0709 13:00:54.352660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.356266 (+  3606us) service_pool.cc:224] Handling call
0709 13:00:54.364494 (+  8228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.440279 (+ 75785us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:54.610083 (+169804us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.508917 (+898834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.565159 (+ 56242us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.614968 (+ 49809us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:55.615044 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.615147 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.733585 (+118438us) raft_consensus.cc:1685] finished
0709 13:00:55.733609 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.733764 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30284,"prepare.queue_time_us":225812,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":76136,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":20218,"thread_start_us":779015,"threads_started":2,"wal-append.queue_time_us":1076640}
I20260709 13:00:55.868574  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.768311  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:55.870327  8824 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:55.873189  9445 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:55.836071 11819 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:55.863313 11390 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.813434 11905 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.761873 11870 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:55.874833 11874 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.891819  8822 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.827631 11855 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.894315 11869 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:55.893955 11875 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.901695  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:55.986320 11895 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:55.904636  8643 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 6 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:55.906416 11844 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:55.910108 11900 raft_consensus.cc:1064] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:00:55.892712 11951 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.887764 11421 consensus_queue.cc:1243] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:55.882623 11892 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:55.969377 11935 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:00:55.828909 11904 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:55.955570  8890 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:55.956236  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:55.959427  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:56.032401  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 73077us
I20260709 13:00:56.026500 11892 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:55.969972  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 201807us
I20260709 13:00:56.017565 11895 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.932144 11907 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:55.895119 11877 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 13:00:56.002926  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 134496us
I20260709 13:00:56.005648  8822 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.969142 11858 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.988860 11819 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.012869  9448 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:00:55.902899  9449 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:55.892508 11794 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:55.914371 11901 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.000798 11882 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.109848 11882 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.112675 11882 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.001849 11605 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.113595 11605 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.116191 11605 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.002381 11449 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.000887 11870 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:55.982534 11889 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:00:55.889966 11854 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.117888 11870 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.118726 11854 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.119400 11870 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.989056 11683 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:00:56.122068  8895 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:55.966941 11893 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:56.123339  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:55.982309 11821 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.124854  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:55.983644  9442 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:00:56.037029  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80883.1us
I20260709 13:00:56.000563 11390 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.010051 11852 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:00:56.023224 11900 raft_consensus.cc:1068] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Unable to promote non-voter 112105225cc140459688be5d31dcb071: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:00:56.040337 11874 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.040896 11895 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:56.156306  8822 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
W20260709 13:00:56.055653  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.058176  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.158310  8891 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:56.058429  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.060137  8894 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:56.017983  8643 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 7, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 7 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.006433 11952 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:00:56.075711  8889 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:56.077451  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.077935 11871 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:56.081851 11768 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:56.019814 11857 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.087342 11875 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.088577 11839 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.049449 11826 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:00:56.089645 11960 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:00:56.090967 11905 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:56.044449 11907 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:56.091665  8646 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 197) took 19603 ms (client timeout 15000 ms). Trace:
I20260709 13:00:56.094123 11595 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:56.095907  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:56.034030  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:00:56.126742  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65882.7us
W20260709 13:00:56.130730  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.131850  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.135049  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:56.059576 11819 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.136029  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:56.137758 11821 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.029589 11904 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.045161 11876 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.026576 11872 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:00:56.160223  8643 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 5 to 6, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.026990 11951 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.182256  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1809) took 1018 ms. Trace:
I20260709 13:00:56.208783  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:56.210542  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:56.234522  8743 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:56.233155  8890 rpcz_store.cc:276] 0709 13:00:55.164192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.167706 (+  3514us) service_pool.cc:224] Handling call
0709 13:00:56.182078 (+1014372us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540046e954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:56.182189 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561515264}
I20260709 13:00:56.237588  8746 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:56.212580  8824 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:56.213179  9440 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:56.140071 11882 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.160162 11874 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.227221  8643 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.242893  8643 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.162690  8892 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:00:56.217142  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81250.2us
I20260709 13:00:56.228966  8895 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:00:56.218572  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 184733us
I20260709 13:00:56.231125  8821 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:56.246435  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:56.223461  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:56.223788  8822 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:56.248968  8643 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 8, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 8 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.251595  8643 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 5 to 7, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 7 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.253350  8643 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 4, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.255788  8643 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.192535 11946 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:56.192238 11879 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:56.191756 11889 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.215926 11854 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.231047 11875 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.232131 11876 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.258152  8643 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 6, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 6 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.268486  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:00:56.355918  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 87576.3us
W20260709 13:00:56.240751 11940 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:00:56.250702 11874 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.225721 11957 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.363556  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:56.378361  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:00:56.329928 11839 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.261081 11857 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.366096 11854 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:56.334304 11952 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.395602  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:00:56.218935  8646 rpcz_store.cc:269] 0709 13:00:36.441560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.441850 (+   290us) service_pool.cc:224] Handling call
0709 13:00:56.026102 (+19584252us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:51.283019 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:51.283830 (+   811us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:51.283893 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:00:51.283925 (+    32us) tablet.cc:662] Decoding operations
0709 13:00:51.360679 (+ 76754us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:51.360773 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:51.360816 (+    43us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:00:51.485386 (+124570us) tablet.cc:701] Row locks acquired
0709 13:00:51.485448 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:00:51.485648 (+   200us) write_op.cc:270] Start()
0709 13:00:51.485826 (+   178us) write_op.cc:276] Timestamp: P: 1783602051485576 usec, L: 0
0709 13:00:51.485850 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:00:51.486675 (+   825us) log.cc:844] Serialized 5893 byte log entry
0709 13:00:52.797708 (+1311033us) op_driver.cc:464] REPLICATION: finished
0709 13:00:53.015380 (+217672us) write_op.cc:301] APPLY: starting
0709 13:00:53.015498 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:53.015838 (+   340us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:53.015966 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:53.108191 (+ 92225us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:53.114566 (+  6375us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:00:53.114625 (+    59us) write_op.cc:312] APPLY: finished
0709 13:00:53.121588 (+  6963us) log.cc:844] Serialized 101 byte log entry
0709 13:00:53.149602 (+ 28014us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:53.161917 (+ 12315us) write_op.cc:454] Released schema lock
0709 13:00:53.171693 (+  9776us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:53.172114 (+   421us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:53.172148 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3764552,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":149956,"apply.run_cpu_time_us":5063,"apply.run_wall_time_us":156743,"num_ops":14,"prepare.queue_time_us":590277,"prepare.run_cpu_time_us":145625,"prepare.run_wall_time_us":1051364,"raft.queue_time_us":324539,"raft.run_cpu_time_us":37730,"raft.run_wall_time_us":215577,"replication_time_us":1269525,"thread_start_us":1551812,"threads_started":4,"wal-append.queue_time_us":774754,"wal-append.run_cpu_time_us":19819,"wal-append.run_wall_time_us":1884579}]]}
I20260709 13:00:56.422569  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:56.418797  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.240406 11855 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:00:56.360365 11876 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.422894  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.425246  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.260428 11869 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:56.361120 11875 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.363044  8643 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 3, term changed from 3 to 5, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to <none>, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129), VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130), VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131), VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.443442  8891 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:56.453948  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.454653  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.454999  8816 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:56.455356  8891 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:00:56.336273 11858 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:00:56.443445  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.252950 11826 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:00:56.447911  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.448258  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:56.465337  8889 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:56.404131 11957 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.452697  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:56.366189 11879 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.471815  8817 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:56.454602  8643 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.331902 11905 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:00:56.388441 11768 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.388095 11897 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:56.331713 11882 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.359870 11889 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.467428  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:56.446194 11931 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.001s
I20260709 13:00:56.452936 11595 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.385435 11852 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:00:56.464604  8892 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:56.360100 11946 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.460368 11875 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.444062  8895 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:56.489149  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:56.490758  8889 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:00:56.460595 11920 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:56.464522 11858 consensus_queue.cc:1243] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:56.399609 11839 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.494613  8890 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.496464  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:56.497650 11847 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:56.498585  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:00:56.766851  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 268478us
I20260709 13:00:56.465607 11903 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.431871 11953 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:00:56.771713 11839 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:56.772382 11819 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.000s
W20260709 13:00:56.513908  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 304233us
I20260709 13:00:56.823464  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:56.824744  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:56.485054 11706 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.892s	user 0.009s	sys 0.002s
I20260709 13:00:56.470124 11869 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:00:56.825596 11706 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.826238 11869 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.827441 11826 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.827843 11826 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.828145 11826 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.516183  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:56.473774 11900 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.829658 11900 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:56.828943 11907 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.001s
I20260709 13:00:56.831930 11900 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.480875 11891 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:56.479022 11854 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.833469 11449 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:56.475097  8643 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 4 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:00:56.562945  8897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.007s	sys 0.024s
I20260709 13:00:56.590986 11390 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.591282  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:00:56.571171  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.514986 11844 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.612416  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:56.484054 11951 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:56.649296  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.654845  8861 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.017s	sys 0.013s
I20260709 13:00:56.655413  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.514688 11955 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:00:56.734692  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.834043  9445 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:56.858376 11706 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.837841 11877 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:56.859156  8891 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:00:56.859120 11877 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.859684  8789 leader_election.cc:400] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6.
I20260709 13:00:56.859997  8789 leader_election.cc:403] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:56.861699  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 249472us
I20260709 13:00:56.861505  8643 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 5, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 5 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:56.862944  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.863039  8643 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:56.864079  8895 raft_consensus.cc:2410] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:00:56.864290  8897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.007s	sys 0.028s
W20260709 13:00:56.864811  8861 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.018s	sys 0.015s
I20260709 13:00:56.864981  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 502) took 1640 ms. Trace:
I20260709 13:00:56.866333  9444 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:56.865334  8897 rpcz_store.cc:276] 0709 13:00:55.224262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.291980 (+ 67718us) service_pool.cc:224] Handling call
0709 13:00:55.292301 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.327458 (+ 35157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.327499 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.336193 (+  8694us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.735089 (+398896us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.017790 (+282701us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.864684 (+846894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.864755 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.864811 (+    56us) raft_consensus.cc:1685] finished
0709 13:00:56.864822 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.864933 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34735,"prepare.queue_time_us":757930,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":53416,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":13779,"thread_start_us":1444732,"threads_started":3,"wal-append.queue_time_us":537123}
I20260709 13:00:56.860884  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:56.869464  8891 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:56.870111  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 381151us
I20260709 13:00:56.870826  8643 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.841436  9442 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:56.875975  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:56.764601  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 650) took 1070 ms. Trace:
I20260709 13:00:56.473438 11935 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:56.666987 11893 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.653164 11595 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.774753  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:56.879381  9447 rpcz_store.cc:276] 0709 13:00:55.694701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.869091 (+174390us) service_pool.cc:224] Handling call
0709 13:00:55.881712 (+ 12621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.881905 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:56.344368 (+462463us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.763909 (+419541us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.764144 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:56.764200 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.764259 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.764369 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.764525 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63557,"prepare.queue_time_us":596765,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":92,"raft.queue_time_us":29691,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":87430,"thread_start_us":772652,"threads_started":2}
I20260709 13:00:56.884575  8895 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:00:56.848806 11870 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.000s
I20260709 13:00:56.872483  8643 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.820391 11931 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.843292 11889 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.873663 11541 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.875232 11706 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 5. Current term is 6.
I20260709 13:00:56.900727  8643 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:00:56.939047  8643 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.943892  8643 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:00:56.950695  9442 pending_rounds.cc:85] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 5
I20260709 13:00:56.951094  9442 pending_rounds.cc:107] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:56.951392  9442 raft_consensus.cc:2889] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:56.883209 11897 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:56.886264 11979 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.887856 11893 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:56.889987 11982 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.889586 11595 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.869562 11819 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.954116  8643 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 1 to 4, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.958428 11889 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:00:56.972831 11979 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.973610 11946 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.836884 11847 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.832655 11891 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.833714 11854 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.832331 11907 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.889716 11768 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.849922 11826 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:56.981962  8892 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:56.890960 11794 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.898931 11978 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.898823 11900 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:56.906876 11905 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.928290 11449 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.936648 11821 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.873867 11844 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.937240 11541 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:56.998626  8893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.007s	sys 0.027s
W20260709 13:00:56.982462 11986 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:00:56.949541  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62647.7us
I20260709 13:00:56.899125 11902 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:00:57.018432 11920 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:56.986737 11893 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:56.998085 11874 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.001s
I20260709 13:00:56.906123 11868 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 13:00:56.985095 11882 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.929198 11931 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:56.992201 11877 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.973204 11955 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:00:57.070760  8891 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:57.058179  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 187409us
I20260709 13:00:57.075137 11794 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.076442 11907 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:56.985569 11897 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:57.050396 11974 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:00:57.049275 11879 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:56.992756 11982 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.137004  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:57.316015  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.130510  8821 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:00:57.284411  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:57.199582 11857 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.000s
W20260709 13:00:57.212615 11962 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.563s	user 0.002s	sys 0.000s
W20260709 13:00:57.208937 11968 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.685s	user 0.004s	sys 0.000s
I20260709 13:00:57.103542 11449 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.199133 11973 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.729s	user 0.002s	sys 0.001s
I20260709 13:00:57.109127 11826 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.393784  8894 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:57.394238  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:57.226212 11981 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 13:00:57.052290 11768 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:57.155871 11990 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:00:57.225152 11977 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.384s	user 0.001s	sys 0.001s
I20260709 13:00:57.326579 11905 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.408152  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 64924.1us
W20260709 13:00:57.212337 11994 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.141s	user 0.001s	sys 0.001s
W20260709 13:00:57.408496  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 124203us
I20260709 13:00:57.455420  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:00:57.114530 11889 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:57.195091 11985 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.280786 11541 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.115830 11852 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.333108 11877 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:57.116107 11953 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.438939  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:00:57.129822 11979 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.391230 11988 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:57.393313 11773 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.394811 11882 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:57.411494 11449 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:57.501957  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 63124.6us
I20260709 13:00:57.502378  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1490) took 1079 ms. Trace:
I20260709 13:00:57.502892  8818 rpcz_store.cc:276] 0709 13:00:56.422990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.423150 (+   160us) service_pool.cc:224] Handling call
0709 13:00:56.453364 (+ 30214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.475559 (+ 22195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.475603 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.476141 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.922052 (+445911us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.922111 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.922209 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.502123 (+579914us) raft_consensus.cc:1685] finished
0709 13:00:57.502150 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.502321 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21739,"raft.queue_time_us":35231,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":20070,"thread_start_us":445548,"threads_started":1,"wal-append.queue_time_us":1020536}
I20260709 13:00:57.465960  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:57.478528  8896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.019s	sys 0.031s
I20260709 13:00:57.457965 11819 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.489357  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.386607 11920 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.490765  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2107) took 1818 ms. Trace:
I20260709 13:00:57.157066 11531 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.443182 11905 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.488171  8747 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:00:57.492878  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:57.333859 11900 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:57.490132 11973 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:00:57.488849  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.492862 11988 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:57.478114 11826 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:00:57.497067  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:57.483229  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:57.517747  8896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.020s	sys 0.031s
I20260709 13:00:57.518721  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:57.483739 11965 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.090s	user 0.002s	sys 0.007s
I20260709 13:00:57.459268 11889 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:00:57.480930 11985 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.480290 11847 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.083576 11978 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.333638 11874 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.458438 11879 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:00:57.595829  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.722471  8749 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:57.723048  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1816) took 1666 ms. Trace:
I20260709 13:00:57.723300  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.539122  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.580752  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 621) took 1360 ms. Trace:
I20260709 13:00:57.724673  8741 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:57.725761  9449 rpcz_store.cc:276] 0709 13:00:56.220420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.220575 (+   155us) service_pool.cc:224] Handling call
0709 13:00:56.220988 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.225189 (+  4201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.225215 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.236142 (+ 10927us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.522325 (+286183us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.522380 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.522474 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.580428 (+1057954us) raft_consensus.cc:1685] finished
0709 13:00:57.580470 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.580652 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3776,"raft.queue_time_us":4282,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8998,"thread_start_us":285843,"threads_started":1,"wal-append.queue_time_us":766010}
I20260709 13:00:57.725728  8746 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:57.575263  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 656) took 1008 ms. Trace:
I20260709 13:00:57.734691  9442 rpcz_store.cc:276] 0709 13:00:56.566521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.893653 (+327132us) service_pool.cc:224] Handling call
0709 13:00:56.950440 (+ 56787us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:56.952140 (+  1700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.952194 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:56.952837 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 13:00:57.123152 (+170315us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:57.123209 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.123303 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.574966 (+451663us) raft_consensus.cc:1685] finished
0709 13:00:57.574990 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.575186 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":941,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":29259,"thread_start_us":141780,"threads_started":1,"wal-append.queue_time_us":435993}
I20260709 13:00:57.715672  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 513) took 1308 ms. Trace:
W20260709 13:00:57.499466 11999 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
W20260709 13:00:57.517566 11977 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:00:57.732757  8823 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
W20260709 13:00:57.497843 11993 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.289s	user 0.000s	sys 0.000s
W20260709 13:00:57.498908 12001 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:00:57.744547  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.524979  8820 rpcz_store.cc:276] 0709 13:00:55.672226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.754697 (+ 82471us) service_pool.cc:224] Handling call
0709 13:00:55.755015 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.755450 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.755486 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.764934 (+  9448us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.145792 (+380858us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.145833 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.145892 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.153356 (+1007464us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:57.444183 (+290827us) raft_consensus.cc:1685] finished
0709 13:00:57.444207 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.490704 (+ 46497us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3909,"raft.queue_time_us":1240,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3931,"spinlock_wait_cycles":471282560,"thread_start_us":364578,"threads_started":1,"wal-append.queue_time_us":663534}
I20260709 13:00:57.733166  8744 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:57.495221 11966 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.034s	user 0.003s	sys 0.000s
W20260709 13:00:57.581221 12003 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:57.484704 11891 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:57.498422 12009 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:00:57.583810 11892 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.541s	user 0.009s	sys 0.002s
I20260709 13:00:57.746506  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:00:57.596639 11875 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.820s	user 0.008s	sys 0.002s
I20260709 13:00:57.748714  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:57.741950  8818 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:57.738595 11768 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:57.597851 11895 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.426s	user 0.011s	sys 0.000s
W20260709 13:00:57.573318 12012 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:00:57.518045 11882 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:57.582834 11595 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.000s
I20260709 13:00:57.755378  8820 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.723424  8896 rpcz_store.cc:276] 0709 13:00:56.057049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.059180 (+  2131us) service_pool.cc:224] Handling call
0709 13:00:56.108441 (+ 49261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.108676 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:56.481522 (+372846us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.722754 (+1241232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.722804 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.722841 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.722910 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.723000 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42122,"prepare.queue_time_us":485286,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":126564,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":25629,"thread_start_us":1603277,"threads_started":3,"wal-append.queue_time_us":1124775}
I20260709 13:00:57.494510 12004 raft_consensus.cc:2764] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:00:57.610018 11876 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.149s	user 0.009s	sys 0.002s
W20260709 13:00:57.615466 11904 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.288s	user 0.012s	sys 0.000s
W20260709 13:00:57.757855  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.51069e+06us
I20260709 13:00:57.524183 11979 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.525161 11935 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.760463  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:57.584589 11794 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.538515 11854 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.598714 11931 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.793944  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 510) took 1387 ms. Trace:
I20260709 13:00:57.766573 11875 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.485405 11852 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.796020 11985 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.796216 11875 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.796329 11852 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.786150 11935 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.777855 11889 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.797506 11935 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.797948 11889 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.798727 11901 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.776307 11895 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.769119  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:57.796900  8817 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:57.788975  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.803323  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:57.802093 11979 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.769951  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 613) took 2090 ms. Trace:
I20260709 13:00:57.761780  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.772811  8749 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:00:57.754329 11531 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:57.773752  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 151383us
I20260709 13:00:57.497416 11877 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.747735 12005 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.818194 11877 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.818360 12005 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.819038  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1826) took 1416 ms. Trace:
I20260709 13:00:57.819372 12017 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.819962 12017 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.819589  8893 rpcz_store.cc:276] 0709 13:00:56.402305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.402407 (+   102us) service_pool.cc:224] Handling call
0709 13:00:56.402733 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.443180 (+ 40447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.443209 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.452721 (+  9512us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.267963 (+815242us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:57.268002 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.268070 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.818814 (+550744us) raft_consensus.cc:1685] finished
0709 13:00:57.818834 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.818969 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21055,"raft.queue_time_us":47159,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4543,"thread_start_us":775517,"threads_started":1,"wal-append.queue_time_us":722428}
I20260709 13:00:57.768958 11891 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.821512 11891 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:57.821835  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2105) took 2233 ms. Trace:
I20260709 13:00:57.821981 11891 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.822201  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:57.776381  8744 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:57.823284  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2109) took 2150 ms. Trace:
I20260709 13:00:57.823402  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.824259  8744 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:57.493562 11953 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:57.777801  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 614) took 2098 ms. Trace:
I20260709 13:00:57.736842  8898 rpcz_store.cc:276] 0709 13:00:56.407605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.888394 (+480789us) service_pool.cc:224] Handling call
0709 13:00:56.888695 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.982532 (+ 93837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.982562 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.983101 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.427918 (+444817us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:57.443975 (+ 16057us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.444209 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.444285 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.715412 (+271127us) raft_consensus.cc:1685] finished
0709 13:00:57.715433 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.715594 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25108,"prepare.queue_time_us":31421,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":7877,"raft.queue_time_us":133385,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":11648,"spinlock_wait_cycles":39407872,"thread_start_us":444549,"threads_started":1,"wal-append.queue_time_us":729953}
I20260709 13:00:57.782683  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:57.825750  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:57.790261 11900 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.825764  9446 rpcz_store.cc:276] 0709 13:00:55.679422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.679556 (+   134us) service_pool.cc:224] Handling call
0709 13:00:55.679934 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.680298 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.680325 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.680782 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.004388 (+323606us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:56.004426 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.004508 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.157837 (+1153329us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:57.773438 (+615601us) raft_consensus.cc:1685] finished
0709 13:00:57.773463 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.777732 (+  4269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5752,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":782542208,"thread_start_us":323274,"threads_started":1,"wal-append.queue_time_us":579659}
I20260709 13:00:57.790781  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 691) took 1294 ms. Trace:
I20260709 13:00:57.791376 11541 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.775341 11844 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.793375 11876 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:57.763993 11892 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.837782 11541 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.838866 11892 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.829764 11852 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.801496 12015 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:57.802047  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.840236  8741 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:57.840123 12015 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.841202  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.841874  8741 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:00:57.842730  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:57.843240  8741 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.843286 12015 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.844043 11895 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.838873 11876 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.787451 11904 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.794368  8897 rpcz_store.cc:276] 0709 13:00:56.406801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.867227 (+460426us) service_pool.cc:224] Handling call
0709 13:00:56.867491 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.994504 (+127013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.994551 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.998967 (+  4416us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.265682 (+266715us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:00:57.387995 (+122313us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.388193 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.388286 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.793718 (+405432us) raft_consensus.cc:1685] finished
0709 13:00:57.793743 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.793879 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20304,"prepare.queue_time_us":122106,"raft.queue_time_us":118740,"raft.run_wall_time_us":14613,"spinlock_wait_cycles":39007232,"thread_start_us":316883,"threads_started":2,"wal-append.queue_time_us":305716}
I20260709 13:00:57.846787 11904 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.771342  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:57.749584  8823 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:00:57.778658 11595 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.823736  8819 rpcz_store.cc:276] 0709 13:00:55.672848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.756459 (+ 83611us) service_pool.cc:224] Handling call
0709 13:00:55.764580 (+  8121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.778366 (+ 13786us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:55.831381 (+ 53015us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.022249 (+190868us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.187509 (+165260us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.207883 (+ 20374us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.208033 (+   150us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:56.208065 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.208129 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.158251 (+950122us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:57.823072 (+664821us) raft_consensus.cc:1685] finished
0709 13:00:57.823099 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.823224 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276222,"prepare.queue_time_us":55713,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3888,"raft.queue_time_us":192135,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":40377,"spinlock_wait_cycles":603003136,"thread_start_us":182202,"threads_started":1,"wal-append.queue_time_us":553954}
W20260709 13:00:57.848664  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66095.8us
I20260709 13:00:57.819150  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 689) took 1923 ms. Trace:
I20260709 13:00:57.822243  8815 rpcz_store.cc:276] 0709 13:00:55.588358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.588510 (+   152us) service_pool.cc:224] Handling call
0709 13:00:55.588952 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.589377 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.589418 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.599322 (+  9904us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.723751 (+124429us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b18002abe60. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:00:56.015344 (+291593us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.015370 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.015414 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.195070 (+1179656us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:57.821603 (+626533us) raft_consensus.cc:1685] finished
0709 13:00:57.821627 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.821767 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1448,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"spinlock_wait_cycles":697416320,"thread_start_us":291424,"threads_started":1,"wal-append.queue_time_us":766558}
I20260709 13:00:57.828537  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.828871  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.822432 11982 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:57.830224  9440 rpcz_store.cc:276] 0709 13:00:56.496319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.496475 (+   156us) service_pool.cc:224] Handling call
0709 13:00:56.496790 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.497214 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.497254 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.498741 (+  1487us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.882869 (+384128us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:56.882905 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.882963 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.790588 (+907625us) raft_consensus.cc:1685] finished
0709 13:00:57.790607 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.790728 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2166,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":383715,"threads_started":1,"wal-append.queue_time_us":623362}
I20260709 13:00:57.787904 11988 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.829545 11931 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.832787 11953 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.833869  8745 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:57.835103 11979 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.835750 11882 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:57.794561  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 653) took 1229 ms. Trace:
I20260709 13:00:57.841544 11852 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:57.822400 12017 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:57.812101  9441 rpcz_store.cc:276] 0709 13:00:55.679132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.681451 (+  2319us) service_pool.cc:224] Handling call
0709 13:00:55.681747 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.788254 (+106507us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:56.215370 (+427116us) log.cc:844] Serialized 14 byte log entry
0709 13:00:56.558216 (+342846us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.591535 (+ 33319us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.591657 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:56.591700 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.761956 (+1170256us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:57.762028 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.762095 (+    67us) raft_consensus.cc:1685] finished
0709 13:00:57.767283 (+  5188us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.769881 (+  2598us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":672787,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2547,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":46977,"spinlock_wait_cycles":243986688,"thread_start_us":653523,"threads_started":2,"wal-append.queue_time_us":729468}
I20260709 13:00:57.850524  8744 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:00:57.787559 11768 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.851253  8749 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:00:57.851850  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.854054  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:57.992816  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:57.993911  8744 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 5: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:57.871042 11847 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:00:57.815390 11854 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:57.849319  9443 rpcz_store.cc:276] 0709 13:00:55.896121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.908747 (+ 12626us) service_pool.cc:224] Handling call
0709 13:00:56.332012 (+423265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.360766 (+ 28754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.360808 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.361274 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.708165 (+346891us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:56.708230 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.708348 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.578878 (+870530us) raft_consensus.cc:1685] finished
0709 13:00:57.578919 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.818917 (+239998us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540065f4d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:57.819092 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28369,"raft.queue_time_us":37224,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":18823,"spinlock_wait_cycles":973005568,"thread_start_us":346519,"threads_started":1,"wal-append.queue_time_us":1133175}
I20260709 13:00:57.875511  9445 rpcz_store.cc:276] 0709 13:00:56.565631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.859008 (+293377us) service_pool.cc:224] Handling call
0709 13:00:56.859334 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:56.859952 (+   618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.859978 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:56.860510 (+   532us) log.cc:844] Serialized 52 byte log entry
0709 13:00:57.046621 (+186111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.046694 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.046825 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.794338 (+747513us) raft_consensus.cc:1685] finished
0709 13:00:57.794358 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.794500 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":185594,"threads_started":1,"wal-append.queue_time_us":284884}
I20260709 13:00:57.873442 11931 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:57.852120 11794 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:57.852098 11875 raft_consensus.cc:2764] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Leader election decision vote started in defunct term 5: won
W20260709 13:00:58.047204 11893 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.000s
I20260709 13:00:58.047720 11893 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.035426  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.053733 11920 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:58.054205  8749 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:58.053622  8821 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:57.882273  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66037.1us
I20260709 13:00:58.053544 11794 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.055852  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:58.060797  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.060809  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:57.866005  8741 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:57.992372 11768 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:58.063558  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 206003us
W20260709 13:00:58.089741  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.439s (SENT) [suppressed 14 similar messages]
I20260709 13:00:58.065196  8895 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:58.099222  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:58.093900 12019 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:58.104288 12018 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:00:58.104084 12019 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.106242 12019 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.107622 11893 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.107949  8742 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:00:58.108314  8742 pending_rounds.cc:85] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 4
I20260709 13:00:58.108525  8742 pending_rounds.cc:107] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:00:58.108738  8742 raft_consensus.cc:2889] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:58.093647 11902 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.109982 11902 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.112062 11902 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.078864 11953 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.113595 11953 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.114068 11953 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.097615 11541 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.117841 11541 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:58.119364  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84491.8us
I20260709 13:00:58.053511 11988 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.122295  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.123028  8895 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.120321  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.125490  8896 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:58.078025 11900 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:58.128329  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.129222  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 503) took 1739 ms. Trace:
I20260709 13:00:58.129757  8890 rpcz_store.cc:276] 0709 13:00:56.390211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.390411 (+   200us) service_pool.cc:224] Handling call
0709 13:00:56.390773 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.667776 (+277003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.128912 (+1461136us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400048dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:58.129146 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238284160}
I20260709 13:00:58.132114  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.132937  8890 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:58.134054  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.135381  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.136134  8890 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:58.137324  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.086562  8741 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:58.078173 12017 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:58.098228 11449 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.001s
I20260709 13:00:58.044342 11854 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.146829  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.149155  8741 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:00:58.150377  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.151748  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:58.086885 11876 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:00:58.168552  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.176757  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:58.181141  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 162060us
I20260709 13:00:58.146351 12026 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:58.146276 12017 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.186022  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.188488  8896 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:58.190936  8895 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:58.191435  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.191910  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.192552  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.193478  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.193691  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.194154  8895 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:58.194660  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.194983  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.195950  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.196410  8744 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 5.
I20260709 13:00:58.197011  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.197811  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.198215  8744 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:58.198813  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.199334  8744 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:58.199865  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.200310  8744 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:58.200875  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.146412 11449 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.146291 11900 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.213517 11891 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.216212 12017 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:58.242019 11794 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.243785 11869 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.246903 11869 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.222820 11854 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.188400 12026 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.227138 11449 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:58.228031 12025 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:00:58.208024 11595 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:58.236507 11541 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.251274 11891 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:58.258610  8896 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:00:58.260295 11935 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:58.260646 11905 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.004s
I20260709 13:00:58.244067 11852 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:58.261919  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.254966  8741 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:58.245391 11768 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.256475 12026 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.264483  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:58.267405  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.260632 11931 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.269197 11900 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.270219  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.266793  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:58.272075  8744 raft_consensus.cc:2410] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:58.301680  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.302934  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.303547  8744 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:00:58.304373  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.304919  8744 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:00:58.305708  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.306316  8897 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.307217  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.308027  8744 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:58.308846  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.309577  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 620) took 2156 ms. Trace:
I20260709 13:00:58.309970  9448 rpcz_store.cc:276] 0709 13:00:56.153024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.153197 (+   173us) service_pool.cc:224] Handling call
0709 13:00:56.153724 (+   527us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:00:56.173180 (+ 19456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.173212 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:00:56.240665 (+ 67453us) log.cc:844] Serialized 78 byte log entry
0709 13:00:56.709004 (+468339us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.709060 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.709147 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.271349 (+1562202us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b24002ddb68. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f3420fb0 00007f90f6097488 00007f90f6097be8 00007f90f606f398 00007f90f606a76a 00007f90f6069bb6
0709 13:00:58.309274 (+ 37925us) raft_consensus.cc:1685] finished
0709 13:00:58.309309 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.309519 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9227,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":31378,"spinlock_wait_cycles":8845568,"thread_start_us":323040,"threads_started":1,"wal-append.queue_time_us":625349}
I20260709 13:00:58.311717  8740 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:58.312915  8740 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:58.315204  8898 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:58.315053  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48347.2us
I20260709 13:00:58.316232  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:58.318351  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:58.319012  8897 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:58.320009  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.320631  8897 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.322439  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:58.322847  8740 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.318804 11704 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.277506 11901 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.314535 11951 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.322149 11847 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.272164 11905 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.330197  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:58.277506 11935 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:58.320076 11920 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:58.333917  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.336516  8744 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 5.
I20260709 13:00:58.337216  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:00:58.324518 11704 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.338196  8898 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:00:58.338439  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.339113  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.339663  8898 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:58.341028  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.335548 11847 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.336774 11920 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.342492  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:58.338335 11704 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.342581  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.343166  8895 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:58.346154  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.330763 11953 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.335731 11901 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.337035 11951 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.353188  8740 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:58.353820  8896 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:00:58.391567  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:58.353710  8744 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.359884  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1487) took 1943 ms. Trace:
I20260709 13:00:58.354625 11920 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.394395  8822 rpcz_store.cc:276] 0709 13:00:56.416645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.416811 (+   166us) service_pool.cc:224] Handling call
0709 13:00:56.417188 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.510901 (+ 93713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.510938 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:56.510975 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.511002 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.359582 (+1848580us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:58.359647 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.359812 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215101696}
I20260709 13:00:58.405709  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:58.357169  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 512) took 1949 ms. Trace:
I20260709 13:00:58.353545  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:58.384373 12017 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.390606 11768 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:58.421322  8890 raft_consensus.cc:2368] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:00:58.353041 11847 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.429243  8744 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:00:58.408895  8891 rpcz_store.cc:276] 0709 13:00:56.407347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.874582 (+467235us) service_pool.cc:224] Handling call
0709 13:00:56.874962 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.284889 (+409927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.356841 (+1071952us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:58.357093 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403365120}
I20260709 13:00:58.431502  8891 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:58.434608  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.480686  8898 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:58.594311 11935 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.595091 11935 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.527508 11905 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.480005 11768 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:58.597745 11449 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.430253 12017 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:58.608493 11891 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.607879 11595 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:59.081570  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2098) took 1186 ms. Trace:
I20260709 13:00:59.082198  8898 rpcz_store.cc:276] 0709 13:00:57.895583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.412032 (+516449us) service_pool.cc:224] Handling call
0709 13:00:58.480384 (+ 68352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.481245 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.081501 (+600256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.081657  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:59.086480  8744 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:59.002995  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.089886  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:59.092140  8740 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.090588  8891 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:00:59.092672  8891 pending_rounds.cc:85] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 3
I20260709 13:00:59.092911  8891 pending_rounds.cc:107] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:00:59.093152  8891 raft_consensus.cc:2889] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:59.093833  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2104) took 1043 ms. Trace:
I20260709 13:00:59.094138  8891 rpcz_store.cc:276] 0709 13:00:58.050379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.500190 (+449811us) service_pool.cc:224] Handling call
0709 13:00:59.090231 (+590041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.093595 (+  3364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.093785 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280}
I20260709 13:00:59.097530  8744 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:59.099466  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:59.102771  8744 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.717521 11901 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.107511  8891 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:00:59.020991 11683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.000s	sys 0.010s
I20260709 13:00:58.739403 11718 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:59.114218  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2108) took 1062 ms. Trace:
I20260709 13:00:59.117573  8898 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:59.118221  8740 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
W20260709 13:00:58.860533 11988 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260709 13:00:59.117729  8891 rpcz_store.cc:276] 0709 13:00:58.051309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.105256 (+1053947us) service_pool.cc:224] Handling call
0709 13:00:59.105662 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.111251 (+  5589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.114143 (+  2892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.118461  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2107) took 1067 ms. Trace:
I20260709 13:00:59.119910  8898 rpcz_store.cc:276] 0709 13:00:58.051102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.101186 (+1050084us) service_pool.cc:224] Handling call
0709 13:00:59.117227 (+ 16041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.118219 (+   992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.118410 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:59.127008  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 362470us
I20260709 13:00:59.127866  8891 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:59.127982  8787 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.003s	sys 0.016s
W20260709 13:00:59.129386  8787 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.004s	sys 0.016s
W20260709 13:00:59.129711  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 729023us
W20260709 13:00:59.130395  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.115039 11683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.000s	sys 0.013s
W20260709 13:00:59.133731  8893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.010s	sys 0.013s
W20260709 13:00:59.134374  8893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.013s
I20260709 13:00:59.109112 11935 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:58.740211 11449 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.140519  8744 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:00:59.116835 11718 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:59.102885 11891 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.125710 11905 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.143224  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.121490 11852 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:59.106786 11595 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.146096  8822 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.146665  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.146915  8744 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:59.147953  8818 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:00:59.148705  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.149452  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 476) took 1035 ms. Trace:
I20260709 13:00:59.149750  8744 rpcz_store.cc:276] 0709 13:00:58.113615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.145382 (+1031767us) service_pool.cc:224] Handling call
0709 13:00:59.146481 (+  1099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.149203 (+  2722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.149411 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.136598 11988 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:59.155457  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.142989 11935 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.156664  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.160645  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 478) took 1046 ms. Trace:
I20260709 13:00:59.161815  8824 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.162719  8897 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:59.160972  8744 rpcz_store.cc:276] 0709 13:00:58.114050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.150405 (+1036355us) service_pool.cc:224] Handling call
0709 13:00:59.160309 (+  9904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.160432 (+   123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.160453 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:59.160482 (+    29us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:59.160499 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.160536 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.160603 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.148324 11905 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.168949 11718 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:00:59.170344  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 516) took 2754 ms (client timeout 2186 ms). Trace:
I20260709 13:00:59.173588  8891 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:59.176546  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2111) took 1124 ms. Trace:
W20260709 13:00:59.175714  8894 rpcz_store.cc:269] 0709 13:00:56.408421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.516183 (+107762us) service_pool.cc:224] Handling call
0709 13:00:59.162441 (+2646258us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.162555 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218406656}
I20260709 13:00:59.177703  8891 rpcz_store.cc:276] 0709 13:00:58.052040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.127326 (+1075286us) service_pool.cc:224] Handling call
0709 13:00:59.127730 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.176345 (+ 48615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.176496 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.161219 12017 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:59.180060  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 762089us
I20260709 13:00:59.151813 11852 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.166502 12023 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.166090 12042 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:59.185119  8744 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.187287  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 479) took 1073 ms. Trace:
I20260709 13:00:59.187508  8744 rpcz_store.cc:276] 0709 13:00:58.114267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.167546 (+1053279us) service_pool.cc:224] Handling call
0709 13:00:59.184977 (+ 17431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.187111 (+  2134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.187246 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.180032 11718 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.162019 11988 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.163419 11935 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.196616  8743 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:59.197165  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 482) took 1082 ms. Trace:
I20260709 13:00:59.197503  8743 rpcz_store.cc:276] 0709 13:00:58.115021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.196172 (+1081151us) service_pool.cc:224] Handling call
0709 13:00:59.196409 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.196930 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.197112 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.200249 11449 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.200376 11882 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:59.200546 11595 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.201313 11595 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.202211  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2097) took 1316 ms. Trace:
I20260709 13:00:59.202204  8744 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:59.204324  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 481) took 1089 ms. Trace:
I20260709 13:00:59.200913 11882 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.205693 11821 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:59.206137 11821 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.206306 11882 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.206822 11882 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.207278 11882 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.203119  8893 rpcz_store.cc:276] 0709 13:00:57.886125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.410996 (+524871us) service_pool.cc:224] Handling call
0709 13:00:58.411398 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.420906 (+  9508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:58.420943 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:58.422986 (+  2043us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.134985 (+711999us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:59.135053 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.135150 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.202021 (+ 66871us) raft_consensus.cc:1685] finished
0709 13:00:59.202039 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.202154 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":606461,"raft.queue_time_us":9168,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":57877,"thread_start_us":582475,"threads_started":1,"wal-append.queue_time_us":662536}
I20260709 13:00:59.209537  8744 rpcz_store.cc:276] 0709 13:00:58.114746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.188180 (+1073434us) service_pool.cc:224] Handling call
0709 13:00:59.201920 (+ 13740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.204101 (+  2181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.204275 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.189117 12023 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:59.239969  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81924.6us
W20260709 13:00:59.239955  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.244360 11449 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.245044 11449 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:59.247617 11449 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.252938  8744 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.209262 11882 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:59.190191 12042 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.264119  8891 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:00:59.266399  8744 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:00:59.267237  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 491) took 1001 ms. Trace:
I20260709 13:00:59.267614  8744 rpcz_store.cc:276] 0709 13:00:58.265392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.254071 (+988679us) service_pool.cc:224] Handling call
0709 13:00:59.266110 (+ 12039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.266907 (+   797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.267171 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.275275 11874 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.203876 11595 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.241662 12044 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.243834 12051 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:59.282683 12051 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.285488  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1513) took 3076 ms. Trace:
I20260709 13:00:59.285854  8742 rpcz_store.cc:276] 0709 13:00:56.208768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.208903 (+   135us) service_pool.cc:224] Handling call
0709 13:00:56.209242 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.109123 (+1899881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.285258 (+1176135us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b5400466ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:59.285433 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4367340416}
I20260709 13:00:59.286685 11852 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:59.287374 12026 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.965s	user 0.010s	sys 0.000s
I20260709 13:00:59.287480 11891 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:59.287688  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 510) took 1765 ms (client timeout 2282 ms). Trace:
I20260709 13:00:59.287814 12026 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.288281 11891 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.290771 12023 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.282589 12044 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.293356 11449 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:59.293841 11449 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.294883 12044 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:59.295770 11449 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.296802 11905 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:59.299165 11595 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.302829  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:59.303157  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.303690  8822 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:59.303934  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:59.304440  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:59.304966  8822 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.302835  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.311084  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:00:59.311704  8819 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.312850  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.313936  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.304970  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:59.314800  9412 connection.cc:256] RPC call timeout handler was delayed by -2.3784555279999609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:00:59.315147 11905 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:59.315466  9412 connection.cc:256] RPC call timeout handler was delayed by -1.7688000570000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:00:59.282586  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:00:59.306258 11920 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.886s	user 0.004s	sys 0.006s
I20260709 13:00:59.316126  8893 raft_consensus.cc:2376] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10. Re-sending same reply.
I20260709 13:00:59.316385 11920 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.317032  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.317670 11907 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:59.318078 11907 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.318257  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.307196  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.318465 11907 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.307934  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.318938  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2099) took 1421 ms. Trace:
I20260709 13:00:59.315629 11905 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.320205 11905 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:59.288219  8817 rpcz_store.cc:269] 0709 13:00:57.522610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.720844 (+198234us) service_pool.cc:224] Handling call
0709 13:00:59.287539 (+1566695us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400076a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.287637 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118686976}
W20260709 13:00:59.321420 11970 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.929s	user 0.016s	sys 0.007s
I20260709 13:00:59.321920  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.322708  8817 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 9 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:59.323734  8817 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:59.309791  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.289157 11891 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:00:59.310422  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.325537  8815 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:00:59.310899 11704 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.891s	user 0.007s	sys 0.005s
I20260709 13:00:59.326574 12051 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.326894 11704 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.318429 11920 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.328979 11704 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.316921 11718 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:59.319051  8894 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:59.319334  8897 rpcz_store.cc:276] 0709 13:00:57.897243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.411519 (+514276us) service_pool.cc:224] Handling call
0709 13:00:59.162557 (+751038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.163677 (+  1120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.318751 (+155074us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b540019e1d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.318885 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166322560}
I20260709 13:00:59.308897  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2103) took 1234 ms. Trace:
I20260709 13:00:59.324967 11951 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.315177  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
W20260709 13:00:59.347209  9415 connection.cc:256] RPC call timeout handler was delayed by -2.1814391539999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:00:59.347796  9415 connection.cc:256] RPC call timeout handler was delayed by -1.545139153999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22820000000000001s for a response
I20260709 13:00:59.347925  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:59.348140  9415 connection.cc:256] RPC call timeout handler was delayed by -1.1686228620000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 13:00:59.348512  8818 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:00:59.308943  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.314857  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.354143  8823 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:59.354395  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2106) took 1303 ms. Trace:
I20260709 13:00:59.308583 12042 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.354954  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.354779  8896 rpcz_store.cc:276] 0709 13:00:58.050873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.393851 (+342978us) service_pool.cc:224] Handling call
0709 13:00:59.314720 (+920869us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:00:59.354337 (+ 39617us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4723584}
I20260709 13:00:59.355707  8823 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.356812  8823 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:59.357545  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.358292  8818 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:00:59.320760 11821 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:59.315933  9412 connection.cc:256] RPC call timeout handler was delayed by -1.4295000569999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2439s for a response
I20260709 13:00:59.361438  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:59.361717  9412 connection.cc:256] RPC call timeout handler was delayed by -1.2523163029999864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 13:00:59.362205  9412 connection.cc:256] RPC call timeout handler was delayed by -1.2100163029999749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
W20260709 13:00:59.362576  9412 connection.cc:256] RPC call timeout handler was delayed by -1.1201349659999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2557s for a response
I20260709 13:00:59.362457  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.363251  8896 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:59.363281  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:59.287099  8742 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.365783  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1814) took 1035 ms. Trace:
I20260709 13:00:59.366178  8742 rpcz_store.cc:276] 0709 13:00:58.330625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.286643 (+956018us) service_pool.cc:224] Handling call
0709 13:00:59.286936 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.365492 (+ 78556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.365721 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:59.342053  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.343364  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.367527  8817 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:00:59.365061  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:00:59.338339  8895 rpcz_store.cc:276] 0709 13:00:58.050118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.194747 (+144629us) service_pool.cc:224] Handling call
0709 13:00:59.284238 (+1089491us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003be054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.284372 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338871680}
I20260709 13:00:59.359581  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.315397 11852 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:59.369724  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.370338  8815 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.345466  8819 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:00:59.336676  8822 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:59.372757  8824 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.370157  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:59.326606 11891 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.310465 11935 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.279980 11874 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:00:59.337779 12023 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:59.399911  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.400507  8893 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:59.386412  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:59.360074 11704 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.388429 11718 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:59.390012 12026 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:00:59.393559 12055 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.380631 12042 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:59.382206 11882 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:59.394436 11907 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.394564 11905 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.395387  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.424467  8889 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:59.424752  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.397027  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.397390 12048 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:00:59.418596  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.426762  8895 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:00:59.429024  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 2.57081e+06us
I20260709 13:00:59.398232 11951 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.396361  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:59.442354  8896 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.439988  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.444351  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2101) took 1394 ms. Trace:
I20260709 13:00:59.444746  8890 rpcz_store.cc:276] 0709 13:00:58.049620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.137054 (+ 87434us) service_pool.cc:224] Handling call
0709 13:00:59.434805 (+1297751us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002099d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.444284 (+  9479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651322112}
I20260709 13:00:59.446269  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1831) took 1713 ms. Trace:
I20260709 13:00:59.440925  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.446648  8892 rpcz_store.cc:276] 0709 13:00:57.732483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.733150 (+   667us) service_pool.cc:224] Handling call
0709 13:00:57.733455 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.815933 (+ 82478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.815962 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.829707 (+ 13745us) log.cc:844] Serialized 26 byte log entry
0709 13:00:58.019258 (+189551us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:58.019323 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.438269 (+1418946us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:59.438340 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.438397 (+    57us) raft_consensus.cc:1685] finished
0709 13:00:59.438414 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.446199 (+  7785us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16530,"raft.queue_time_us":11002,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":164092672,"thread_start_us":179943,"threads_started":1,"wal-append.queue_time_us":264172,"wal-append.run_cpu_time_us":2498,"wal-append.run_wall_time_us":1150247}
I20260709 13:00:59.452687  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.420172 12055 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.423273 12008 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.464234  8896 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:00:59.398948 11855 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:00:59.426867 11882 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:00:59.456852  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72071.4us
I20260709 13:00:59.420436 12042 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:00:59.398376 12045 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:00:59.422187 11704 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.419379 12023 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.442206 11951 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:59.474167  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 104038us
I20260709 13:00:59.494843  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.410173 11874 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.445991 11872 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.497227 11872 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.500151 11872 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.475979  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:00:59.456554  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:00:59.455799 11891 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.501809 11891 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.476352 12042 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:59.502305  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:59.462467 12055 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.446923 12026 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 13:00:59.475764 11847 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.832s	user 0.003s	sys 0.006s
I20260709 13:00:59.481900 12007 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:59.475579 11882 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.478263 12050 raft_consensus.cc:2764] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:00:59.447391 12048 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.505786 11951 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:59.543109  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122243us
I20260709 13:00:59.513195 12017 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.572425  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 492) took 1255 ms. Trace:
W20260709 13:00:59.575439  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 200670us
I20260709 13:00:59.533891 12065 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:59.573103  8744 rpcz_store.cc:276] 0709 13:00:58.316997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.268813 (+951816us) service_pool.cc:224] Handling call
0709 13:00:59.330645 (+ 61832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.330875 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:00:59.361175 (+ 30300us) log.cc:844] Serialized 14 byte log entry
0709 13:00:59.571966 (+210791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:59.572035 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:00:59.572097 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.572189 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.572339 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11931,"prepare.queue_time_us":38457,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":34,"raft.queue_time_us":30045,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":192007,"threads_started":2,"wal-append.queue_time_us":147919}
I20260709 13:00:59.510996 11847 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.544344 12008 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:59.587648 12026 consensus_queue.cc:1243] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:00:59.599887 11869 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:00:59.606667  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101460us
W20260709 13:00:59.609094  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 95414.3us
I20260709 13:00:59.510142 12007 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.576062 12068 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:59.585263 12045 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.575480 12017 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:00:59.630244 12080 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:59.630821 12074 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:59.631826 12080 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:59.632310 12080 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:00:59.614250 12026 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:59.656478 11976 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.816s	user 0.010s	sys 0.020s
I20260709 13:00:59.581670 12066 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 13:00:59.684263  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.626662 12054 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:00:59.689555  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 106782us
I20260709 13:00:59.690358  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:00:59.689030  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112643us
I20260709 13:00:59.691828  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:00:59.624735 12068 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:00:59.693981  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1802) took 2205 ms (client timeout 1680 ms). Trace:
I20260709 13:00:59.694099  8749 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:59.658100 12026 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:59.701638  8745 rpcz_store.cc:269] 0709 13:00:57.488823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.489067 (+   244us) service_pool.cc:224] Handling call
0709 13:00:59.693658 (+2204591us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54004646d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:00:59.693912 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710226816}
I20260709 13:00:59.691128 12008 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:00:59.621830 12065 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:00:59.693105 12045 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:59.799683  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 188695us
I20260709 13:00:59.621796 12081 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.169s
W20260709 13:00:59.783859 12089 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:00:59.815759  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.720676 12087 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.841941  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1807) took 2027 ms. Trace:
I20260709 13:00:59.800532 12045 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:00:59.855979 11980 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 3.016s	user 0.012s	sys 0.020s
I20260709 13:00:59.813967 12007 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.858417 12017 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.858935  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1510) took 3901 ms. Trace:
I20260709 13:00:59.859810 12068 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.860387 12008 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.862749 12085 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:59.850226 12064 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:00:59.867390  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.904405  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 210671us
W20260709 13:00:59.856061 12063 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:00:59.877914 11978 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.878016 11891 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:00:59.890111 12051 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:59.818753 12023 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:00:59.850409 11976 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:00:59.884351  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.885036  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.922464  8744 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:59.885736  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.885998 12005 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.886364  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:59.923712  8747 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.887008 11899 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.888353 11897 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.889634  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 655) took 3323 ms. Trace:
I20260709 13:00:59.926719  9444 rpcz_store.cc:276] 0709 13:00:56.566239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.867819 (+301580us) service_pool.cc:224] Handling call
0709 13:00:56.868067 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.074356 (+206289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.074385 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.074427 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:57.074460 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.889409 (+2814949us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400660dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:00:59.889455 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.889579 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420679552}
I20260709 13:00:59.889097 12080 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:00:59.889885 11449 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:00:59.893253  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 495563us
I20260709 13:00:59.943573  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.944389  8745 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:00:59.894335  8748 rpcz_store.cc:276] 0709 13:00:55.957789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.958976 (+  1187us) service_pool.cc:224] Handling call
0709 13:00:55.994734 (+ 35758us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:55.995167 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.995207 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:56.015871 (+ 20664us) log.cc:844] Serialized 52 byte log entry
0709 13:00:56.510672 (+494801us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:56.510714 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.510781 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.855230 (+3344449us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:00:59.858654 (+  3424us) raft_consensus.cc:1685] finished
0709 13:00:59.858674 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.858879 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6075,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":6714,"spinlock_wait_cycles":66973056,"thread_start_us":467381,"threads_started":1,"wal-append.queue_time_us":810182}
I20260709 13:00:59.849836  8740 rpcz_store.cc:276] 0709 13:00:57.814125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.121073 (+1306948us) service_pool.cc:224] Handling call
0709 13:00:59.121499 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.121958 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.121985 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.137358 (+ 15373us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.353504 (+216146us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:59.353540 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.353600 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.841635 (+488035us) raft_consensus.cc:1685] finished
0709 13:00:59.841667 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.841859 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8856,"thread_start_us":193876,"threads_started":1,"wal-append.queue_time_us":290109}
I20260709 13:00:59.906328  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:59.860450 12078 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:00:59.879087 12083 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:00:59.910912  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.911989  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.912177  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.967598  8896 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.914208  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:00:59.968010  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.968297  8815 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:00:59.917627  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:00:59.969400  8742 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:00:59.918677  8822 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:00:59.970219  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.970299  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:59.910717 11847 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:00:59.908221 12079 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
W20260709 13:00:59.973933 12082 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:59.919747  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.928834  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:00:59.931058  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.936182  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:00:59.846907 11983 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 3.006s	user 0.009s	sys 0.028s
I20260709 13:00:59.945240  8820 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:00:59.945077  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:00:59.947163  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 256953us
I20260709 13:00:59.960726 11897 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:00:59.962427 12086 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.966540  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.971493  8818 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:59.916924 12048 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:00:59.911100 12068 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:00.020202  8746 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.021086  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:00.021466  8819 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:00.022652  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:00:59.945899 12066 consensus_queue.cc:1243] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:00.018213  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.024619  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:00.024871  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:00.031795  8897 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:00.024641  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:00:59.977926  8890 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:00.025961  8894 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:00.037705  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:00.038544  8889 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:59.969169 12051 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:00:59.981335  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.039942  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 238905us
I20260709 13:01:00.045015  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.049448  8745 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:59.981559  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 98117us
I20260709 13:01:00.054685  8746 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:01:00.055718  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.055886  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:00.071544  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.980772  8747 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:01:00.056094  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.072291  8744 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:00.056535  8746 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:00:59.971662 11953 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:00.057655  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.057997  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.029937 12080 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.061529  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:00:59.920974 12087 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
W20260709 13:01:00.067970  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.053453  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:00.052479 11852 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.034s	sys 0.025s
I20260709 13:01:00.072913  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.075846  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:00.077024  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.081820  8742 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:00:59.942886 11899 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.115867  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:00.115032  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 92232.6us
W20260709 13:01:00.118321  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.119402  8740 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:00.112601 11624 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:01:00.082535 12017 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.120898 12017 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:00.121266 12017 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:00.124761 11882 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.000s
I20260709 13:01:00.125483 12077 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:00.035001 11897 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.129338 12096 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:00.129858 12017 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:00.130980  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54056.9us
I20260709 13:01:00.130939  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:00.131752  8744 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:00.132894  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:00.133752  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.117627  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:00.069976 12066 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.136102  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:00.139292  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 223180us
I20260709 13:01:00.145524  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:00.146641  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:00.147269  8890 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:00.147344  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:00.149734  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:00.130738 11852 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.038s	sys 0.029s
W20260709 13:01:00.149995  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 99480.5us
I20260709 13:01:00.153894  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:00.072846  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 616582us
I20260709 13:01:00.038681 12068 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.153554  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.155104  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.039651 12086 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.083473 12053 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:00.158613  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.164474  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.164801  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.165529  9447 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:00.159305  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.166982  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.167181  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.169389 11869 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:00.169761 11869 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:00.170749 11869 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.171965 11869 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.166060  9443 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:00.160930  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:00.155659  9445 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:00.161633  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.061861 12091 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:01:00.163053 12085 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:00.176188  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.193485  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:00.193801  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.177775  9449 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:00.194492  9441 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.195796  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.198194  8892 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:00.178110  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.180852  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:00.177364  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.178562  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.125206 11882 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.141520 12081 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:01:00.154197 11852 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:00.212262  9445 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:00.194346  9443 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:00.209132  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.163807 11901 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:00.164062 12053 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.205989  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.213176  9444 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:00.213138  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.230532  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:00.231316  9443 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:00.231225  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.231984  9445 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:00.179309 12077 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.173872 12072 raft_consensus.cc:2764] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:00.186046 12090 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:00.234041  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:01:00.214911  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:00.217453  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62770.1us
I20260709 13:01:00.193732 11897 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:00.186594 11624 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.257364 12113 raft_consensus.cc:2764] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:00.207590 11449 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:00.173529 12051 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.207921 12048 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:00.342604  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 208967us
W20260709 13:01:00.351800  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.462378  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.463213  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.460362  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.232434 12049 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:00.230923 11901 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.467365 12113 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:00.459471 12104 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:01:00.466905  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.464248  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.236951 12053 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.477334  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.459082 11624 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.483510  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:00.352967 12100 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.342895 12008 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:00.352887 11852 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.275192 12077 raft_consensus.cc:2764] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:00.444465 11821 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:00.447952 12017 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:00.484588  9441 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:00.278227 12090 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:00.299772 12101 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.481681  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.475028 11901 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:00.468392 11872 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:00.474885 12113 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.475970 12048 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:00.556315 12101 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:00.601635 12101 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.600755 12113 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.601069 12090 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.535722 12117 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:00.485875 12098 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
W20260709 13:01:00.490880  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 330296us
I20260709 13:01:00.556864 12049 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:00.600970 11882 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:00.628250  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.634773  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1524) took 2850 ms. Trace:
W20260709 13:01:00.645434  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.645710  8749 rpcz_store.cc:276] 0709 13:00:57.784784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.313815 (+529031us) service_pool.cc:224] Handling call
0709 13:00:59.693891 (+1380076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.781820 (+ 87929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.634511 (+852691us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54004646d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:00.634689 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3174325632}
W20260709 13:01:00.641098  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.534291  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.555796  9443 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:01:00.557899  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74485.9us
I20260709 13:01:00.490025 11624 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.285385 12072 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.231s
I20260709 13:01:00.605075  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 505) took 3140 ms. Trace:
I20260709 13:01:00.616009  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2125) took 1426 ms. Trace:
W20260709 13:01:00.639505 12042 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.135s	user 0.011s	sys 0.000s
I20260709 13:01:00.612663  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:00.608505 12117 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:00.686196  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 481587us
W20260709 13:01:00.630221 11951 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.092s	user 0.007s	sys 0.004s
I20260709 13:01:00.599268  8898 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:01:00.602777 12105 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:01:00.610288 11857 raft_consensus.cc:1064] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:01:00.774230 12052 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:00.776242 11857 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.748802 11935 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:00.604898 11704 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:00.637821 12068 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:00.734091  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:00.745632  8821 rpcz_store.cc:276] 0709 13:00:57.465126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.465318 (+   192us) service_pool.cc:224] Handling call
0709 13:00:57.493200 (+ 27882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.054783 (+561583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.604833 (+2550050us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:00.605019 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288845568}
I20260709 13:01:00.746330  9441 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:00.534759  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.746502  8891 rpcz_store.cc:276] 0709 13:00:59.190009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.190144 (+   135us) service_pool.cc:224] Handling call
0709 13:00:59.190392 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.265315 (+ 74923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.615757 (+1350442us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b5400094554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:00.615950 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168627968}
I20260709 13:01:00.746873  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:00.683334  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 219196us
I20260709 13:01:00.504508 12115 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.790647  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.791280  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:00.792383  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 179824us
I20260709 13:01:00.779625  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:00.795840  8712 leader_election.cc:400] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:01:00.796279  8712 leader_election.cc:403] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:00.793391  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:00.801903  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55140.8us
I20260709 13:01:00.576288 12017 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:00.620453 11905 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:00.704957 12106 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.358s	user 0.000s	sys 0.005s
I20260709 13:01:00.636052 11907 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:00.625900 11899 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:00.624223 11847 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:00.797713 12115 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.763062 12042 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.722112  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:00.784046  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:00.779347  8890 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.787698  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:00.849814  8892 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:00.777242 11869 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.004s
W20260709 13:01:00.805224  8741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 485) took 2664 ms (client timeout 1991 ms). Trace:
I20260709 13:01:00.618798 12049 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:00.852210  8741 rpcz_store.cc:269] 0709 13:00:58.140534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.176518 (+ 35984us) service_pool.cc:224] Handling call
0709 13:01:00.805029 (+2628511us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b5400358c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:00.805159 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178448768}
I20260709 13:01:00.559865 11872 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:00.814782  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.815881  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.777459 11390 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:00.779454 11870 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:00.798732 12065 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:00.820645 12116 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:01:00.820473 11704 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:00.821564 11683 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:00.807018 12072 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:01:00.826095 12119 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:00.827627  8821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:00.648912 11449 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.611115 12048 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:00.778540 11935 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:00.763921 12117 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.494345 12008 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.493314 12100 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.778189 11768 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:00.840117  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.962844  9447 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:00.794024 11874 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:00.843201  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49965.1us
I20260709 13:01:00.779702 11951 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.615162 12045 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.846418  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.964895  9449 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:00.820470 12050 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.809626 12126 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:00.717396  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:00.977599  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 260367us
I20260709 13:01:00.621063 12101 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:00.534044 12055 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.021s	user 0.009s	sys 0.001s
I20260709 13:01:00.560253 12077 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:00.862711 11821 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.846439 11891 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.847047 11899 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.842613 11847 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:00.868031  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:00.992359  8891 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:00.849822 12115 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:00.873013 12103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.016s
I20260709 13:01:00.960590  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.919725  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:00.880287  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.973011  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.872731 11872 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:00.883814 12084 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.004810  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.005998  9443 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:00.986711 12077 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:00.889050 12042 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:00.853964 12113 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.887169 12052 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:00.963124 11683 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:00.966794 12119 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:00.883060 12090 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.999027  9441 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:00.999193  9447 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:00.976586 12126 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:00.880187 12114 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:01:00.888711 12066 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.001s
I20260709 13:01:00.966928 11624 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:00.985010 11768 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:00.873471 11870 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:00.886328 12124 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:00.998178 12103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.000s	sys 0.020s
I20260709 13:01:00.959081 12122 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.124s
I20260709 13:01:00.992002 11891 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:01.001973 12080 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.000s
I20260709 13:01:01.003441 12084 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:00.987100 12111 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:01:00.986959 12055 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.050454 11891 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.011901 12077 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.039494 12008 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.037400 12090 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:01.006397 12053 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.514s	user 0.014s	sys 0.000s
I20260709 13:01:01.036890 12126 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.049263 12100 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.048868 11988 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.051532 12124 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:01.013227 12044 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.014701 11935 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.043895 12066 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.073204 12017 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:00.853021  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65462.3us
W20260709 13:01:01.126997  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 323723us
I20260709 13:01:00.963950  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:01.128242  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 334951us
I20260709 13:01:01.135483  8821 raft_consensus.cc:606] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:01.135763  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 351754us
I20260709 13:01:01.136325  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.136914  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.137923  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.138274  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:00.979560  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.256856  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:01.257395  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 277847us
I20260709 13:01:01.257869  8823 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:01.139762  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.908298  8890 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:01.000265  8892 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:01.139775  8818 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.001439  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.007892  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:01.280354 11901 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.622s	user 0.006s	sys 0.017s
I20260709 13:01:01.280625  9442 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.281766  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:00.991559 11821 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.282384  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.284736 11821 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.285990  8824 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:01.295055 11592 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:01.078218 12045 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.053045 12084 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.296773 11592 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:01.073120 12044 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:01.310034 11920 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:01.310914 12044 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.311550 12074 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:01.312224 12074 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.312922 11592 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:01.313750 12074 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.314173 11592 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.315308  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1534) took 3067 ms. Trace:
I20260709 13:01:01.092775 12115 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.073796 12119 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.084686 12124 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.318701  8743 rpcz_store.cc:276] 0709 13:00:58.247733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.201739 (+954006us) service_pool.cc:224] Handling call
0709 13:01:00.805639 (+1603900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.125828 (+320189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.125932 (+   104us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.127295 (+  1363us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.163811 (+ 36516us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:01.166199 (+  2388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.313916 (+147717us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b54002dec20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d2a02 00007f90f64cfdde 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:01.314027 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.314084 (+    57us) raft_consensus.cc:1685] finished
0709 13:01:01.314101 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.314913 (+   812us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400358c50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.315039 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54327,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2105,"spinlock_wait_cycles":3693627008,"thread_start_us":353527,"threads_started":2,"wal-append.queue_time_us":41333}
I20260709 13:01:01.311512  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.329704  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.330560  8821 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:01.037233 11847 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.178052  9448 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:01.178306 12090 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.175271 11951 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.188161  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.193056 11624 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.193753 11683 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.194751  9446 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:01.181043 12140 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.206408  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.066044 11870 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:01.210755  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:01.211486  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 203697us
I20260709 13:01:01.216632  8815 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.219303 12026 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.230587  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.229234 12055 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.259171  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.261667  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.256137  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.274750 11858 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.343379 11858 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.343519  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:01.343834 11858 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.344693  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.345602  9444 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:01.346561  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.348124  9444 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:01.348645  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 675) took 1184 ms. Trace:
I20260709 13:01:01.349874  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.351176  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.351594 11683 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.352388  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.353679  8815 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:01.354537  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:01.274940  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 311047us
I20260709 13:01:01.275267 12049 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.275818 12017 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.357664 12049 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.358098 12049 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.362825  9443 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:01.363461  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 693) took 1198 ms. Trace:
I20260709 13:01:01.363790  9443 rpcz_store.cc:276] 0709 13:01:00.164998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.361919 (+1196921us) service_pool.cc:224] Handling call
0709 13:01:01.362369 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.363227 (+   858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.363412 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:01.365098 11899 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:01:01.366461 11870 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.368340 12049 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.279567  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:01.370460  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 91271.1us
I20260709 13:01:01.371147 12049 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.276557  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.372298  8818 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:01.374245  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112904us
I20260709 13:01:01.332294 11847 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.376406 11624 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:01.298523  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:01.377067 11624 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:01.377707  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 79419us
I20260709 13:01:01.378197 12115 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.378472  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:01.378960 12115 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:01.379918 12136 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.300187  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.380540 12136 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.380690  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.381111 12136 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.381421 12140 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.382292  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 666) took 1221 ms. Trace:
I20260709 13:01:01.382649  9446 rpcz_store.cc:276] 0709 13:01:00.160626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.161379 (+   753us) service_pool.cc:224] Handling call
0709 13:01:01.382095 (+1220716us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.382234 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225745152}
I20260709 13:01:01.383723  9446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858"
dest_uuid: "112105225cc140459688be5d31dcb071"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:33086
I20260709 13:01:01.384428  9446 raft_consensus.cc:606] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.384943  8822 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.077275 12086 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:01.308215  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.306838 12084 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.386001  9441 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:01.312268 12044 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.386475  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 702) took 1174 ms. Trace:
I20260709 13:01:01.386679 12026 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.386826  9441 rpcz_store.cc:276] 0709 13:01:00.211985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.307705 (+1095720us) service_pool.cc:224] Handling call
0709 13:01:01.386386 (+ 78681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:01.081992 12101 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.300925  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:01.281144 11901 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:01.389240  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 130015us
I20260709 13:01:01.083094 12113 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.070737 12053 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.391723 11901 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.391953  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.317670 12066 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.392540 12053 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.319321 11592 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.393007 11951 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.393405 12086 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.393550 11951 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:01.043071 11768 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:01.331645  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.394106 11951 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.393008 12066 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.395749 12126 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.394477 11768 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.397955 12126 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.398080 12066 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.399117  8820 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.300026  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.401965  9447 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:01.403687  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.404513  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:01.393882 12086 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:01.405658 12086 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.395198  8821 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:01.406759  9440 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.406968  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.407124  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.317878  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:01.407749  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91977.2us
I20260709 13:01:01.368844  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 659) took 1223 ms. Trace:
I20260709 13:01:01.408391  9448 rpcz_store.cc:276] 0709 13:01:00.145136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.145251 (+   115us) service_pool.cc:224] Handling call
0709 13:01:01.368572 (+1223321us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001982d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.368765 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340074752}
I20260709 13:01:01.409041  8890 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:01.377710 11624 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.410633 12117 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.412835 12117 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.252947 12077 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.379773 12115 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:01.348948  9444 rpcz_store.cc:276] 0709 13:01:00.164459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.347434 (+1182975us) service_pool.cc:224] Handling call
0709 13:01:01.347832 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.348423 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.348575 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:01.419662  9442 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:01.414273  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:01.424795  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 52110.9us
I20260709 13:01:01.425448  9443 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:01:01.425844  9443 pending_rounds.cc:85] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 6
I20260709 13:01:01.429210  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:01.430029  9447 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.431917  8818 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.433561 11891 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.434775 11882 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:01.435376 11882 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.436091 11872 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.436381  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.438067  8819 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.438488 11872 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.436101 11891 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:01.440469  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.441995 12066 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.443490 12066 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.443835 12066 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:01.444922 12066 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.446342 12066 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.450515  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.451483  8818 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.436444 11882 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.426131  9443 pending_rounds.cc:107] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:01:01.423745  8787 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.383723 12140 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.456475  9443 raft_consensus.cc:2889] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:01.465562  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.466563  8822 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:01.466487  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.468060  8819 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.468421 12074 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.468948  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:01.471308 11768 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.471630 12136 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.480443  9440 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:01.484898  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:01.491075  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49489.3us
W20260709 13:01:01.495549  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.496789  8892 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:01.497164  8817 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.493451  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:01.519584  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 139076us
I20260709 13:01:01.521198  8821 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.531931  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.533406 11390 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.533993 11390 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.536092 11390 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:01.537225  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.538499  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60842.1us
I20260709 13:01:01.539472  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:01.541208  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.541703 12125 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.544620 12052 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:01.543787  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 126859us
I20260709 13:01:01.546561  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.498811 11624 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.548110  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.549265 11935 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.549788 11935 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:01.550271 11935 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.550388 11624 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.524688  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 705) took 1046 ms. Trace:
I20260709 13:01:01.523315  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:01.561074  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63457.2us
I20260709 13:01:01.567379  8821 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:01.491217 11872 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.582646  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.588349  8819 raft_consensus.cc:2368] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
I20260709 13:01:01.546940 11858 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.595331  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 622) took 1002 ms. Trace:
I20260709 13:01:01.597409  9446 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:01.597695  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.598495  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 695) took 1432 ms. Trace:
I20260709 13:01:01.598886  9443 rpcz_store.cc:276] 0709 13:01:00.165951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.364772 (+1198821us) service_pool.cc:224] Handling call
0709 13:01:01.414606 (+ 49834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.475879 (+ 61273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.598283 (+122404us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b54006801d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.598430 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113683200}
I20260709 13:01:01.599653  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 699) took 1402 ms. Trace:
I20260709 13:01:01.599961  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 706) took 1115 ms. Trace:
I20260709 13:01:01.597538  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.597849  9446 pending_rounds.cc:85] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 7
I20260709 13:01:01.602880  9446 pending_rounds.cc:107] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:01.603253  9446 raft_consensus.cc:2889] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:01.600805  9447 rpcz_store.cc:276] 0709 13:01:00.478337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.402616 (+924279us) service_pool.cc:224] Handling call
0709 13:01:01.424195 (+ 21579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.432458 (+  8263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.593538 (+161080us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b540065f4d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.593734 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40044928}
I20260709 13:01:01.554328  9441 rpcz_store.cc:276] 0709 13:01:00.468929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.387650 (+918721us) service_pool.cc:224] Handling call
0709 13:01:01.388011 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.388268 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:01.389230 (+   962us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.514085 (+124855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.514576 (+   491us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:01.514635 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.514723 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.514870 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":394,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":134,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":80,"thread_start_us":107019,"threads_started":1}
I20260709 13:01:01.608762  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:01.591060 12136 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.610599  8819 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.637652  8819 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:01.638939  8819 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.596297  9448 rpcz_store.cc:276] 0709 13:01:00.593112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.409272 (+816160us) service_pool.cc:224] Handling call
0709 13:01:01.433174 (+ 23902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.433616 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:01.433944 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.594464 (+160520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.594534 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:01.594723 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.594940 (+   217us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.595144 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18496,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":15048,"raft.queue_time_us":15154,"thread_start_us":144888,"threads_started":1}
W20260709 13:01:01.611822  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 236014us
I20260709 13:01:01.612386  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 625) took 1018 ms. Trace:
W20260709 13:01:01.613773  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82029.6us
I20260709 13:01:01.614276 11852 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:01:01.616676  8821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44b3244bf12f42aba79421575a96c621"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:01.617462 11855 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:01:01.655593 11855 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.624109 11858 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.610138 11390 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.627046 11768 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.604030 12044 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.656464  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.656180  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.656488 11390 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:01.657852 11390 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:01.630424  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 89362.5us
W20260709 13:01:01.631279  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.632606 11891 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.635284  8823 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:01.641875  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.641471  9446 rpcz_store.cc:276] 0709 13:01:00.594049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.432786 (+838737us) service_pool.cc:224] Handling call
0709 13:01:01.497253 (+ 64467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.605814 (+108561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.612063 (+  6249us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54006720d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:01.612303 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129419008}
I20260709 13:01:01.643719 12077 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.649814  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:01.650712  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:01.651505  9414 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.991s (SENT)
W20260709 13:01:01.652078  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47907.1us
I20260709 13:01:01.628912 11951 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.658758  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:01.662806  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 63942.8us
I20260709 13:01:01.656199 11855 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.668479  8821 raft_consensus.cc:606] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.668972  8821 raft_consensus.cc:614] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:01.669631  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.670326  9441 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:01.668151 12125 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.672262 12136 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.672466  8817 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:01:01.674052  8817 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.661415 11891 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.675951  9448 raft_consensus.cc:2410] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:01.675084  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.684751 11855 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:01.685652 12077 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.686540  8820 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:01.600334  9445 rpcz_store.cc:276] 0709 13:01:00.197245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.384414 (+1187169us) service_pool.cc:224] Handling call
0709 13:01:01.384784 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.385161 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:01.413769 (+ 28608us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.596773 (+183004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.597061 (+   288us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:01.599178 (+  2117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.599324 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.599527 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3516,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":65,"raft.queue_time_us":355,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"thread_start_us":181790,"threads_started":1}
I20260709 13:01:01.679803 11891 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.713620  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:01.719333 12117 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:01.719612  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:01.722121  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.723865 12125 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.722637 12049 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.725466 12049 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:01.726127 12049 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.726222 12026 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:01.727443 12026 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:01.727790 12026 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.728220  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:01.724676 12074 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.725806  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.656118 12086 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:01.724910  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73482.2us
W20260709 13:01:01.738641  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47778.2us
I20260709 13:01:01.740293  9448 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:01.720563 12117 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.744637 12136 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:01.761278  9442 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.763592  8892 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.761404  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.766567  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.765906  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:01.769608  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.771363  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.772264 11920 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.772715 11920 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:01.773132 11920 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.773469 12090 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:01.773573  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:01.774116 12090 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:01.774348  8823 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.775197 12090 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.778501  9445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815"
dest_uuid: "112105225cc140459688be5d31dcb071"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:33086
I20260709 13:01:01.779240  9445 raft_consensus.cc:606] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.779646  9445 raft_consensus.cc:614] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:01.780313 12086 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.786619  8823 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:01.776677 11390 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.793118 12045 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.792915 12026 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.794459 12026 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.795166 11390 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.788301 11882 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:01:01.790484  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:01.791052  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.800833  9445 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:01.811072  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 633) took 1214 ms. Trace:
I20260709 13:01:01.811975  9442 rpcz_store.cc:276] 0709 13:01:00.596842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.464005 (+867163us) service_pool.cc:224] Handling call
0709 13:01:01.485993 (+ 21988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.762461 (+276468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.810693 (+ 48232us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54006a95d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:01.810950 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608205568}
I20260709 13:01:01.812754  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.822130  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:01.823128 11882 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:01.823220 12117 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.823834 11882 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.825002 11882 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.826160 12124 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:01.828135 12124 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:01.828830  9445 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:01:01.834164  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 91635.6us
W20260709 13:01:01.834873  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51827us
I20260709 13:01:01.836597  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:01.843894  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.710350 11935 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:01.853518  8892 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.857311  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:01.859992  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 206555us
I20260709 13:01:01.828876 12124 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:01.862298  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:01.864939  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.869601  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.874742  9449 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:01.876308  9442 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:01.895291 12148 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.220s	user 0.004s	sys 0.000s
I20260709 13:01:01.901068  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.902712 11704 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:01.903357 11704 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.904325 11704 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.909413  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:01.912647 12045 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:01.915838 12045 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:01.912845 12157 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:01:01.913393 12042 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:01.919777 12042 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.920409 11935 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:01.921123 12042 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.922009  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 710) took 1360 ms. Trace:
I20260709 13:01:01.922412  9449 rpcz_store.cc:276] 0709 13:01:00.561226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.281240 (+720014us) service_pool.cc:224] Handling call
0709 13:01:01.921816 (+640576us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:01.921951 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361865472}
I20260709 13:01:01.923789 12125 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:01.924180 12125 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:01.925055 12125 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.938673 12065 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:01:01.939765  8891 pending_rounds.cc:85] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 4
I20260709 13:01:01.940199  8891 pending_rounds.cc:107] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:01.940712  8891 raft_consensus.cc:2889] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:01.941684  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:01.940145  8898 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:01.942584  8897 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:01.943902  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:01.945310  8894 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.933013 11870 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:01:01.946970 11870 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:01.947624 11870 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:01.950178  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:01.952224 12086 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:01.933673 12045 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:01.954905 12045 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.956018  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 623) took 1362 ms. Trace:
I20260709 13:01:01.939881 12065 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.957396  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:01.956445  9444 rpcz_store.cc:276] 0709 13:01:00.593430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.417664 (+824234us) service_pool.cc:224] Handling call
0709 13:01:01.478356 (+ 60692us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:01.533756 (+ 55400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.533787 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:01.534441 (+   654us) log.cc:844] Serialized 52 byte log entry
0709 13:01:01.675525 (+141084us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:01.675580 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.675666 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.955743 (+280077us) raft_consensus.cc:1685] finished
0709 13:01:01.955762 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.955923 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4329,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":140594,"threads_started":1,"wal-append.queue_time_us":418914}
I20260709 13:01:01.934187 12125 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.965423  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 634) took 1368 ms. Trace:
W20260709 13:01:01.980273  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 88560.8us
I20260709 13:01:01.864389 11920 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:01.983546  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.986660  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.961126  8647 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:01.961804 11951 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:01.963871 12077 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.959208 12065 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:01.948938 11870 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:02.013046  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 318505us
I20260709 13:01:02.015281  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.025600  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:01.996085 12136 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.005774 12086 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.025816  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.026355  8897 raft_consensus.cc:2376] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:02.030975  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:02.032789  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.012456 11390 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.038035  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:02.040373  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.030843  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:01.990562 11891 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.033066  8895 raft_consensus.cc:2376] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:02.038812  8647 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:02.049226  9442 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:01:02.056442  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.078694  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 676) took 1264 ms. Trace:
I20260709 13:01:02.040162 11872 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:02.085604  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:02.054661 11870 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.086359  8647 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:02.040601 12086 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.126462  8647 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 5, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 5 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:02.085981 11872 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.043598 11390 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.080135 12074 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:02.132066  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101449us
I20260709 13:01:02.133320  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:02.133759  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.134020  8643 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 284) took 11029 ms. Trace:
I20260709 13:01:02.135464  9442 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 LEARNER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:02.136834  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.144372  8741 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:02.145573  8749 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:02.145646  8897 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:02.146451  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.147537  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.148406  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.149513  8745 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:02.149638  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.150128 12065 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.181792  8744 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:02.151587 12086 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.152786 11891 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.153645 11920 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.192333  8743 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9.
I20260709 13:01:02.154760  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.052224 12049 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.124409 11951 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:02.158840  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.082598 12136 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.189378  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:02.231168  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.173260  8894 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 5.
I20260709 13:01:02.184633  8748 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:02.185810  8747 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:02.187304  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 635) took 1341 ms. Trace:
I20260709 13:01:02.188001  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:02.188714  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:02.189950  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.192066  8889 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:02.154079  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.192143  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:01.986671  9440 rpcz_store.cc:276] 0709 13:01:00.597185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.481075 (+883890us) service_pool.cc:224] Handling call
0709 13:01:01.484620 (+  3545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.582607 (+ 97987us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:01.610689 (+ 28082us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.964788 (+354099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.964859 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:01.965019 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.965114 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.965269 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25640,"prepare.queue_time_us":20901,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":30652,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15800,"thread_start_us":303489,"threads_started":1}
I20260709 13:01:02.233736  8816 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:02.348315  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.362133  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.358279  8897 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:02.365422  8746 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.228569  8896 raft_consensus.cc:2376] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10. Re-sending same reply.
I20260709 13:01:02.232632  8818 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:02.438675  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.276813  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.261660  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.318962  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.374123  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:02.236369  8819 raft_consensus.cc:2368] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10.
I20260709 13:01:02.443740  9449 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:02.234813  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.257344  9445 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.318867  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.323065  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.448668  8749 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9.
I20260709 13:01:02.448899  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.453442  8824 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 6: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:02.401088 12167 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:01:02.319495 12165 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:01:02.454964  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:02.456319  8895 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:02.257545 12065 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.455857  8741 raft_consensus.cc:2376] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6. Re-sending same reply.
I20260709 13:01:02.382543 11882 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.319196 12077 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.330191 12086 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.460419  8890 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.323499 12066 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.459663  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:02.459264 12174 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:01:02.493197  8748 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 6.
I20260709 13:01:02.365607 12142 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:02.228684 11951 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:02.359329 11855 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:01:02.267011  8647 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:02.508282  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.232446  9447 rpcz_store.cc:276] 0709 13:01:00.846050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.662987 (+816937us) service_pool.cc:224] Handling call
0709 13:01:01.683926 (+ 20939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.684341 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:01.684719 (+   378us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.170358 (+485639us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.186771 (+ 16413us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.186995 (+   224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:02.187045 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:02.187098 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.187159 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.187254 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19172,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":2625,"raft.queue_time_us":5859,"raft.run_cpu_time_us":491,"raft.run_wall_time_us":53381,"spinlock_wait_cycles":12300544,"thread_start_us":469524,"threads_started":2,"wal-append.queue_time_us":399778}
I20260709 13:01:02.501019 12111 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:02.501194 12176 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:01:02.460879 12045 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.500851 12142 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.504984 11951 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.507529  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 133564us
I20260709 13:01:02.514313  9445 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:02.387791 12055 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
W20260709 13:01:02.535835  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.537529 11390 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.538228 11390 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:02.538830 12183 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:01:02.538684 11390 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.540400 12136 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:02.540548  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:02.540762 11390 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.541200  8889 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:02.542220  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:02.542802  8889 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:02.543783  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.544497  8897 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.545644  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.547816  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:02.549820  8893 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:01:02.553038 12055 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:02.553489 12055 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:02.554265 12055 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.555408 12055 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.557274 12142 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.559674 12045 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.560174 12045 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.562340 12045 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.563037  8647 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 8, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:02.564656  8647 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 6, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:02.571938  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:02.572644  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.577163  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 386872us
I20260709 13:01:02.494566 12077 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:02.581229  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.583837  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:02.498188 12086 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.530745 11935 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.496075 11882 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:02.586014  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 400030us
I20260709 13:01:02.595013  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.610224  8822 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.575832 12167 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:02.561163 11872 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.610556  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:02.614547  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86426us
I20260709 13:01:02.615334  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.618511  8743 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:02.567072 12066 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.175366  8643 rpcz_store.cc:276] 0709 13:00:51.091899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.093490 (+  1591us) service_pool.cc:224] Handling call
0709 13:00:51.505165 (+411675us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b5400002210. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33ab640 00007f90fc282b76 00007f90fc38cc3e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:01:02.087860 (+10582695us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:57.210316 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:57.210961 (+   645us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:57.210995 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:00:57.211013 (+    18us) tablet.cc:662] Decoding operations
0709 13:00:57.409355 (+198342us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:57.412734 (+  3379us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:57.417425 (+  4691us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:00:57.601529 (+184104us) tablet.cc:701] Row locks acquired
0709 13:00:57.601558 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:00:57.601753 (+   195us) write_op.cc:270] Start()
0709 13:00:57.601903 (+   150us) write_op.cc:276] Timestamp: P: 1783602057601691 usec, L: 0
0709 13:00:57.601934 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:00:57.602811 (+   877us) log.cc:844] Serialized 8282 byte log entry
0709 13:00:58.185672 (+582861us) op_driver.cc:464] REPLICATION: finished
0709 13:00:58.339559 (+153887us) write_op.cc:301] APPLY: starting
0709 13:00:58.340517 (+   958us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:58.360689 (+ 20172us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:58.407655 (+ 46966us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:58.706840 (+299185us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:58.707316 (+   476us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:00:58.707386 (+    70us) write_op.cc:312] APPLY: finished
0709 13:00:58.707966 (+   580us) log.cc:844] Serialized 145 byte log entry
0709 13:00:58.709056 (+  1090us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.709416 (+   360us) write_op.cc:454] Released schema lock
0709 13:00:58.709497 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:58.709963 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.710002 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17126,"reported_tablets":47,"spinlock_wait_cycles":13334912,"child_traces":[["op",{"apply.queue_time_us":121561,"apply.run_cpu_time_us":10357,"apply.run_wall_time_us":373436,"num_ops":21,"prepare.queue_time_us":129778,"prepare.run_cpu_time_us":22633,"prepare.run_wall_time_us":603645,"raft.queue_time_us":310903,"raft.run_cpu_time_us":6215,"raft.run_wall_time_us":115289,"replication_time_us":568874,"thread_start_us":285300,"threads_started":4,"wal-append.queue_time_us":223781,"wal-append.run_cpu_time_us":40543,"wal-append.run_wall_time_us":1970156}]]}
I20260709 13:01:02.594856 12142 raft_consensus.cc:476] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Not starting pre-election -- already a leader
I20260709 13:01:02.531611 12111 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:02.565582 12042 consensus_queue.cc:1175] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "29c7934ee528454785432732d01ed0a0" responder_term: 10 status { last_received { term: 10 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 10 index: 10 } }
W20260709 13:01:02.671114  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51818.1us
I20260709 13:01:02.672927  8748 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:02.677613  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53050.7us
I20260709 13:01:02.682418  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:02.619011 11870 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.621609 11390 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.670050  8647 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 7, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:02.686189  8819 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:02.619333 12045 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.687515  8897 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:02.693459  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.693682  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:02.694265  8743 raft_consensus.cc:2368] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
I20260709 13:01:02.612510 11935 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:02.680052 12167 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:02.700860  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.706753  8892 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:02.708768  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.705277 11920 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:02.710829  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.722409  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.723242  8743 raft_consensus.cc:2368] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7.
I20260709 13:01:02.689000 11592 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:02.705921 12125 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.001s
I20260709 13:01:02.734836 11855 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.735463  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.742408  8745 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:02.736038  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.729679  8647 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:02.693452 11870 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.737398  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.744974  8816 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:02.727005  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44829us
I20260709 13:01:02.744396  8819 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:02.750984  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.751549  8823 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.748334  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:02.756361  8743 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.736922 11951 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.758353  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.761307  8817 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:02.758834  8820 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:02.759218  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:02.760407  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 220550us
I20260709 13:01:02.708668 12101 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:01:02.698720 11390 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.079161  9443 rpcz_store.cc:276] 0709 13:01:00.813946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.657804 (+843858us) service_pool.cc:224] Handling call
0709 13:01:01.658188 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.658459 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:01.658759 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.078182 (+419423us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.078362 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:02.078409 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:02.078444 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.078518 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.078635 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13425,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":2838,"raft.queue_time_us":14101,"thread_start_us":404835,"threads_started":1}
I20260709 13:01:02.774591  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.779392  8743 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:02.777429  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:02.786895 11882 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.793810  8745 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:02.796927  8892 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:02.799465  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:02.765669 12189 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:02.754287 11855 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.825269  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:02.837759  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90155.8us
I20260709 13:01:02.838367  8744 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
I20260709 13:01:02.826345  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.775936 11592 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:01:02.805850  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.814623  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.846968  8743 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:02.807641  8647 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:02.848433  8749 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:02.793730 12124 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:02.809279  8740 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:02.816776  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:02.818969  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.802202  8890 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:02.835217  8823 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:02.845234  8816 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:02.817456  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.828083 12189 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:02.852249  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.853242  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.855444 11855 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.861215  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:02.872011  8741 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:02.827600 12186 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:01:02.870424  8745 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:02.806180 12111 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.870424  8744 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:02.874739  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.807492 11935 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.803293 11951 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.823988 12049 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:02.876883  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:02.882134  8746 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.882350  8744 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:02.883997  8746 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:02.885584  8746 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.835529  8748 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:02.893466 11855 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.896522 11951 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.897070 11951 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.899672 11951 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.900781 12189 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.902206 12189 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.856137 12086 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.826918 12191 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:01:02.846529 12192 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:01:02.908494 12192 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:02.908878 12192 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:02.909539 12124 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:02.909940 12192 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.911072 12124 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:02.911567 12124 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:02.911713 12192 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.913170  8647 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:02.912742 12124 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:02.915176  8647 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:02.916728 12124 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:02.882334 12187 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:01:02.919222  8745 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:02.922845 12077 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.925837 11870 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:02.928470 12117 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:02.886498  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.932495  8749 raft_consensus.cc:2410] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:02.933501  8647 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 7, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:02.929016 12117 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:02.940081  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:02.936352  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:02.945988  8741 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:02.958005  8743 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:02.985474  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.988212  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 276649us
I20260709 13:01:02.947427 12066 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:02.993520  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:02.983533 12055 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:03.003540  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.005376  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:03.002640  8647 catalog_manager.cc:5697] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 7, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:03.006734  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66793.1us
I20260709 13:01:03.010097  9415 leader_election.cc:400] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10.
I20260709 13:01:03.010533  9415 leader_election.cc:403] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.014434  9447 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 10. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 13:01:03.017385  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:03.017545  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:03.019594  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.019822  9448 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:03.004426 12055 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:03.007933 11870 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:03.026983 11870 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:03.027985 11870 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.029419 12193 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:03.031913  8741 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:03.041476 12188 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.042353 12188 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:02.996778 12117 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.020987 12111 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.016983 12065 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.046635  8647 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:03.059540  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.066116  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.066984  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.067734  8746 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:03.067960  8741 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:01:03.068598  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:03.069474  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 74172.5us
I20260709 13:01:03.069617  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.069865  8741 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:03.070298  8745 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:03.070571  9414 leader_election.cc:400] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9.
I20260709 13:01:03.070895  9414 leader_election.cc:403] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.071281  8741 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:03.071422  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:01:03.071815  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.072400  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.067636 12125 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:03.071820  8745 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 3
I20260709 13:01:03.069271  8746 raft_consensus.cc:2368] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:03.108597  8745 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:01:03.109104  8745 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:03.049863 12055 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.113546 12111 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9.
I20260709 13:01:03.114580 12111 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:03.115180 12111 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:03.048938 11935 consensus_queue.cc:1175] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 7 status { last_received { term: 7 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 7 index: 8 } }
I20260709 13:01:03.046542 12117 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.122159  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.123003  8748 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:03.128346  8789 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.673s (SENT) [suppressed 41 similar messages]
I20260709 13:01:03.116361 12111 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:03.134752  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.135879  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.138536 11882 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.139199 11882 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.048889 12042 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:03.142921 12042 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.148233  8647 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 1 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:03.152369  8740 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.157121  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.159709  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
W20260709 13:01:03.154584  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 351229us
I20260709 13:01:03.045251 12188 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:03.163421  8743 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:03.165779  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.166253  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.166591  8741 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:03.178143  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.077113 12125 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:03.185001  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 173476us
W20260709 13:01:03.187606  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.188232  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.183369  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.191596  8740 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:03.108909 11704 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:01:03.198047  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.177752 11592 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:03.174424 12189 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.213634  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.218271  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.218164 12136 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.195667 12074 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:03.222370  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:03.227682  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:03.230000  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.212638 12188 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.230108  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.190603 12192 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:03.241762  8647 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 5, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 5 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:03.246981  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:03.212369 12077 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.255474  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.257228  8892 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.223106 12042 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.264459 12055 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.259076  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:03.261590 11872 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.229652 12136 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.233475 12193 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:03.271579  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.279219  8894 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:03.274338  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:03.280961  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 152688us
I20260709 13:01:03.281162  8897 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:03.250792  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:03.279362  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 203575us
I20260709 13:01:03.282843  8743 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:03.283842  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.289644  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.292279  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.292577  8897 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:03.293926  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.294883  8890 raft_consensus.cc:2368] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:03.220521 11592 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.284677  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.297606  8896 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.222528 12189 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.299505  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.301996  8893 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:03.302228  8897 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.299505  8741 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:03.315265  8894 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:03.316557  8889 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.316795  8890 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:03.319383  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60392.1us
I20260709 13:01:03.233932 12074 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.326206  8897 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:03.326524  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.298018 12117 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:03.271288 12077 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.259229 12188 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.343122  8897 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:03.344434  8892 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:03.349972  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.350481 12195 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.158s
I20260709 13:01:03.350800  8890 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 10. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20260709 13:01:03.350672  8897 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:03.351956 12201 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:03.352362  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:03.352679 12201 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:03.350582  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.355266  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.355402 12203 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:03.356173 12197 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:03.353901 12201 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.367462  8740 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:03.368048  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 121170us
I20260709 13:01:03.369771  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.370656  8890 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:03.379736  8894 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:01:03.356412 12203 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:01:03.356182  8863 leader_election.cc:400] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
I20260709 13:01:03.387620  8863 leader_election.cc:403] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.387003  8824 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:03.399381  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:03.401018  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61517us
I20260709 13:01:03.403015  8863 leader_election.cc:400] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7.
I20260709 13:01:03.403569  8863 leader_election.cc:403] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.404173  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.407704  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.408350  8894 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:03.414669  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.402590 12195 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
W20260709 13:01:03.422266  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 69995.5us
I20260709 13:01:03.425689 12124 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.427323  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.443204  8745 raft_consensus.cc:2368] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10.
W20260709 13:01:03.430797 12209 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:01:03.431164 11390 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:03.431584 12211 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:03.446887  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.442709 12124 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:03.447662 12124 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.448015  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.450081  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.450755  8747 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:01:03.432219 12204 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:01:03.451790  8747 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.424255 12196 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7.
I20260709 13:01:03.432672  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.453204  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:01:03.433656 12205 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:03.433386 11935 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:03.454532 12189 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:03.437611 12200 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:01:03.461758  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.438341  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.438838 12192 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:03.464627  8816 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:03.443034  8896 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:03.430161 12206 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:01:03.424685 12222 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:03.423141 12197 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:03.473449  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2140) took 1304 ms. Trace:
I20260709 13:01:03.473919  8824 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.473858  8891 rpcz_store.cc:276] 0709 13:01:02.169252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.169380 (+   128us) service_pool.cc:224] Handling call
0709 13:01:02.184049 (+ 14669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.689508 (+505459us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:02.689547 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.705496 (+ 15949us) log.cc:844] Serialized 26 byte log entry
0709 13:01:03.160605 (+455109us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:03.160645 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.160851 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.473178 (+312327us) raft_consensus.cc:1685] finished
0709 13:01:03.473209 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.473371 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32057,"raft.queue_time_us":45200,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"thread_start_us":785788,"threads_started":2,"wal-append.queue_time_us":512205}
I20260709 13:01:03.463887  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.475401  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.476198  8824 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:03.476626  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.455219 11935 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:01:03.465536  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.463306  8889 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:03.469405  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.477645  8821 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:03.488585  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.477280  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.471962 12222 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:03.481022  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 501) took 1394 ms. Trace:
I20260709 13:01:03.470741  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:03.491988  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.445583 12136 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.494475  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.495850  8824 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:03.480949 12045 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:03.501076  8819 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:03.500773 12045 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.502931 12124 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.501432 12227 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:03.503662 12188 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:03.491060  8742 rpcz_store.cc:276] 0709 13:01:02.114140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.163617 (+ 49477us) service_pool.cc:224] Handling call
0709 13:01:02.164133 (+   516us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:02.164668 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:02.164985 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.741429 (+576444us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:03.445202 (+703773us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b2c00048b68. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:03.462662 (+ 17460us) log.cc:844] Serialized 78 byte log entry
0709 13:01:03.462849 (+   187us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:03.463096 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.463285 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.463351 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.480591 (+ 17240us) raft_consensus.cc:1685] finished
0709 13:01:03.480659 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.480842 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3034,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":2813,"raft.queue_time_us":141038,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":6557,"spinlock_wait_cycles":23920000,"thread_start_us":575937,"threads_started":2,"wal-append.queue_time_us":431027}
I20260709 13:01:03.504931 12124 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.505465 12045 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.506485  8822 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:03.497320 12203 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.510917 12203 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:03.511369 12203 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.511699 12065 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10.
I20260709 13:01:03.513576 12225 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:03.497505 11390 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:03.516412  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.514060 12225 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:03.484724 12195 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:03.563959 12222 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.481141 11704 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:03.481072 12042 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:03.483376 11592 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:03.570541  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:03.563365  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 151896us
W20260709 13:01:03.579114  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55673.8us
I20260709 13:01:03.581493  8863 leader_election.cc:400] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10.
I20260709 13:01:03.581996  8863 leader_election.cc:403] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.583475  8787 leader_election.cc:400] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:03.583988  8787 leader_election.cc:403] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:03.585094  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.525425 12197 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:03.518008 12074 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.567840 12195 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.575322 12065 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.574486 12228 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.595921  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.595681 12066 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:03.587930 12196 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10.
I20260709 13:01:03.567705 11855 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:01:03.598410  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.572716 12192 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:01:03.574308 12124 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:01:03.590809 12189 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.592985 12045 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.591796 12136 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:01:03.611260 11872 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:03.612735 12226 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:01:03.594901 12197 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:03.594573 12195 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:03.594450 12198 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:03.634210 12198 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.634629 12198 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.635344 12049 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:03.636044 12049 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:03.633265  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.637632  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.642856 11882 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.611703  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.578174 11704 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.650319 11704 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.622979 12077 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:03.651818 12219 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:01:03.621460 12045 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:03.659427  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.587519 12042 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:03.669788  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75612.8us
I20260709 13:01:03.633249 12236 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:03.652292 12208 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.227s	user 0.000s	sys 0.002s
I20260709 13:01:03.682753  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.616322 12232 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.646929 12197 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.625530 12196 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.647956 11872 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.602707 12065 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.604067 12228 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.621546 12233 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:03.671578  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:03.683784  8893 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
W20260709 13:01:03.659530  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.725090  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53618.3us
I20260709 13:01:03.714720 11872 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.686174 12111 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.684918 11870 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:03.690687 12196 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.734611 11872 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.741078 12196 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:03.745191  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 250895us
I20260709 13:01:03.745553 12235 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:01:03.746272 12228 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.748363 12065 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.740823 12244 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:03.749462 12238 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:03.751410  8864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.036s	sys 0.028s
I20260709 13:01:03.691756 12232 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:03.774188  8864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.036s	sys 0.028s
I20260709 13:01:03.776631  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.776707  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 538) took 1583 ms. Trace:
I20260709 13:01:03.778455  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.779793  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:03.777644  8820 rpcz_store.cc:276] 0709 13:01:02.187544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.189754 (+  2210us) service_pool.cc:224] Handling call
0709 13:01:02.261871 (+ 72117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.762865 (+500994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.770460 (+1007595us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:03.770683 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120583040}
I20260709 13:01:03.739244 11891 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:01:03.786540 12117 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:03.787258 12240 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:03.784972 12236 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:03.788676  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 1.17844e+06us
I20260709 13:01:03.793304  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.793545 12232 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.794267  8893 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:03.794807  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.794939 11920 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:03.781001 12124 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:03.740428 12111 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.811730  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.739377 12233 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:03.813390  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:03.814244  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.821552  8896 raft_consensus.cc:2376] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7. Re-sending same reply.
I20260709 13:01:03.817878 12045 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.826583  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:03.824410 12228 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:01:03.813225 11920 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:03.842777 12111 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.848052 12227 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:03.837718  8864 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.850677  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.850718 11872 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.851706 12237 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:03.852054  8864 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.852437 12237 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.842156 11920 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.853958 11920 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.854461 12042 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:03.858172 11855 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:03.859395  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:03.847412  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 171155us
I20260709 13:01:03.826192 12236 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:03.836759 12220 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
I20260709 13:01:03.861231 12227 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:03.864606 12228 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.867166  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:03.853127 11872 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:03.872912  8789 leader_election.cc:400] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:03.875272  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:03.875949  8822 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.876168  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.876793  8745 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:03.877087  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.877822  8740 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:03.877698  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:03.878264  8744 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:03.836095 12244 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:03.875845  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.881930  8741 raft_consensus.cc:2376] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9. Re-sending same reply.
I20260709 13:01:03.836995 12246 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:03.883409  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.887372  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.867753 12198 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.890081 12195 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.891361 12198 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:03.892592 12195 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:03.887246  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.894748  8822 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:03.895561 11855 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.898164  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.902689  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:03.897022 12239 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:03.918592  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:03.893321 12237 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:03.914032  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:03.913694  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.921243  8741 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.897342 11855 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.908812  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.909488  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:03.950718  8746 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:03.951185  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.903183 12198 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.893713 12228 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:01:03.904582 12246 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:03.976305  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70078.8us
I20260709 13:01:03.918972 12042 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.920861 12195 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.905313 12227 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:01:03.954999 11855 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.963322 12193 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:03.950417 12111 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:03.995608  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 88232.4us
I20260709 13:01:03.869161 12236 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:03.956290 12244 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.989966 11872 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:03.999852 11872 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.002245 11872 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.004485 12196 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:03.908176 12233 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:01:03.993947  8742 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:03.993999  8743 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.998380 12185 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.977931 12241 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:04.005435  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.012156  8742 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:03.915669 12203 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.015034 12236 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:04.015604 12202 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.745s	user 0.003s	sys 0.000s
I20260709 13:01:04.049976  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:03.985743 12195 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.060626  8890 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.056708  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.011680 12207 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.068713  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.071077  8897 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:04.071388  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.010318 12193 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:04.071967  8898 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:03.992547 12042 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.075412  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.075613  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.077086  8893 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:04.085808  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:04.080128  9444 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:04.075212  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.010528 12111 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.122915  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:04.121824  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:04.098205  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.080886  8742 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.121793  9442 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:04.122112  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:04.065879 12195 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:04.120100  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 217490us
I20260709 13:01:04.147188  9445 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:04.121029  8895 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:04.147392  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.147488  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:04.123768  9449 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:04.049937 12124 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:01:04.150487  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:04.071172 12201 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.146548  8742 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:04.071552 12241 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:04.071799 12045 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:04.151670  9446 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.087808 12065 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.075294 12236 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.152820  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:04.177651  9440 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.152820  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:04.155568  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 136105us
W20260709 13:01:04.144275 12202 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:04.155593  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:04.072090 11920 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:01:04.159297 12254 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:01:03.992352 12246 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.148770 11847 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.161391 12242 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:01:04.161867 12193 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.163215  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:04.164740 11592 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.167337 12189 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:04.171415 12203 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.173239 12241 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.174232 12125 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.158617 11870 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.184508  9442 raft_consensus.cc:2376] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9. Re-sending same reply.
I20260709 13:01:04.183408 12244 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.185420 12196 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.187381  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:04.168978 12225 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.180029 12045 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.184458  8742 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:04.189678  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.190100  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.216415 11592 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:04.226759 12189 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:04.193681  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:04.071934 12232 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:01:04.194373  8741 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:04.197145  9440 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:04.202441  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:04.251672  8748 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.203517  8895 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 9. Ops: []
W20260709 13:01:04.204071  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 56689.5us
W20260709 13:01:04.217587  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.238399  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.191784 12065 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.200793 12124 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:01:04.191022  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.193599 12236 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.245759  8893 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:04.253275  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63695.3us
I20260709 13:01:04.262639  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.254050  9442 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.215093 11847 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:04.268277  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 74674.9us
I20260709 13:01:04.268584  8746 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:04.248329 12045 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:04.268708 12225 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:04.298002  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.301113  8748 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:04.277622 12066 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.309509 11855 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.310693  8896 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.273986 12244 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:04.303642  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:04.311916  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58081.7us
I20260709 13:01:04.318578  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:04.311764  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49225.5us
W20260709 13:01:04.312430  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 121538us
I20260709 13:01:04.320497  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.278375 12055 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.325153  9440 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:04.325640  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.311717 12227 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.332875  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.334028  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.335525  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.336676  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:04.339632  9442 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.252352 12255 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:01:04.326259 12244 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:04.344203  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.332523 12235 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.348980  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.355314  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.346166 12066 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:04.345261 12228 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.370512  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:04.372038  8748 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.370864  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:04.326464 11592 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.358592 12237 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:04.345559 12242 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:01:04.361212 12077 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.358359 12196 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.364651 12198 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.382385 12065 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:04.382534 12257 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 13:01:04.382390 12244 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:04.385722 12268 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:01:04.386633 12272 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:04.392541  8748 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:04.393533  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.333823 12189 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.382158 12263 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:04.414970  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.384246 12278 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.379032 12235 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:04.434962  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.429924 11592 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.382479 12266 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.440028 12198 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:04.441772  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71461.4us
W20260709 13:01:04.441964  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 109172us
I20260709 13:01:04.393481 12237 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:04.445957  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:04.438499  9442 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:04.439145 11855 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.452603  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:04.463732  9440 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.463904  9445 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:04.461690  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.443538 12263 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.465852 12263 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.467370 12198 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:04.469964 12275 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:04.471096  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.472213 12125 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.472266 12237 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.474062 12045 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.474925 12233 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.475347 12233 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:04.475744 12233 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.447522 12189 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.451242 12266 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:04.476083  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.485081  8891 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:04.444520 11870 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:04.488111  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 94721.3us
I20260709 13:01:04.489225  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:04.490072  8740 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:04.450150 12236 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.490939 12066 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:04.491194  8747 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 9. Ops: []
I20260709 13:01:04.491104  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.492309 12277 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:04.492594 12227 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.530645  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.492985 12286 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:04.494184  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:04.534890  8749 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.492236  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:04.495890  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:04.485736 11847 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:04.498126  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:04.501360 12282 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:04.495213 12117 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.507829  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.509127  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.509640  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.511037  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.511354  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.512799 12233 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.517612  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:04.517791  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:04.451493 12196 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.546612  8744 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:04.546988  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 82031.5us
I20260709 13:01:04.547676  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.527184  8818 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:04.529045  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.492420 12280 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:01:04.548960  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.451310 12264 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.535545  8740 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:04.497351 12045 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:04.550738  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.551097  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.552186  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.489235 12242 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:04.556699  8864 leader_election.cc:400] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8.
I20260709 13:01:04.488462 12266 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.558965  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.539198  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.544459  8745 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.545812 12288 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:01:04.565522  8818 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.567269  8821 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:04.547813  8744 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:04.524477 12252 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:04.542286 12117 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.535748 12227 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.554546 12277 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:04.555783  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:04.555859  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57845us
I20260709 13:01:04.556599 12283 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:04.543629  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:04.561576  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.572306  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.451630 12235 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.574064  8824 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:04.581497 12283 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:04.590912 12283 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.581991 12281 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.593643 12066 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.599200  8744 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.573527 12117 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.569624 12197 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.608556  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.608703 12298 raft_consensus.cc:2764] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:01:04.570971 12252 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:04.619594 12291 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.556912 12242 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:04.571275 12045 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:01:04.574339 12065 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:04.612911 12188 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.657967  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.599287 12195 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.642231 12197 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.611773 12193 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:04.627290 11870 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.672927 12227 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.676096  8889 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:04.677759  8817 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:04.665277 12207 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.686394  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.650399 12298 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:04.668694 12197 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:04.695981 12242 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.697806 12233 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:04.698596  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 138626us
I20260709 13:01:04.699313  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:04.693842 12197 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.663288 12277 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.693620 12291 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.706436 12195 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.699324  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.712519  8821 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:04.712580  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:04.714560  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.714532  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.719197  8819 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.720096  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:04.712127 12302 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:01:04.724584  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49077.9us
I20260709 13:01:04.725625  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:04.726606 12195 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:04.727137 12195 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:04.728039 12195 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.718593 12193 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.723285 12283 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:04.740713  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 111736us
I20260709 13:01:04.744038  8745 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:04.732306 12298 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.676589 12252 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.735270 12193 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:04.792733  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74924.7us
I20260709 13:01:04.814457  8893 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.746521 12283 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.816491  8895 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:04.818305  8895 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:04.818585  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.819294  8898 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:04.820001  8896 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:04.820521  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.818320  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:04.821924  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.823951  8891 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.824309  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.825245  8898 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:04.826946  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.808375 12298 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.804147 12197 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.797017 12235 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.813817 12193 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.814456 12252 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.840969  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.835701  8898 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.811838 12263 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.839242 12298 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:04.845532  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.854298  8820 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:04.868371  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55253.6us
W20260709 13:01:04.869067  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 127467us
W20260709 13:01:04.868542  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 159903us
I20260709 13:01:04.870823 12111 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:04.882705 12197 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.856032 12207 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.862833 12313 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:04.884621  8891 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:04.884577 12313 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.886976  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:04.887362 12313 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.886768  9445 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 4 to 7, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:04.890202  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.892206  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.893235  8898 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.897080  8893 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:04.903935  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.904667  8893 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:04.931541  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 217163us
I20260709 13:01:04.932657  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.933545  8893 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:04.933682  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:04.854071 12298 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.929605 12142 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:04.945393 12142 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:04.946209 12142 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.947265 12322 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:01:04.948603 12263 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:04.946558 11870 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:04.951361 11870 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:04.943677  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.903518 12117 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:04.951982  8893 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:04.953226  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 83865.9us
I20260709 13:01:04.955171 12287 raft_consensus.cc:2764] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:04.896742 12266 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:04.959439  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.961093  8818 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:04.958104  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.965447  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:04.966693  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:04.967028  8742 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.972110  8743 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:04.972654 12291 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.975180  8815 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:04.963914 12263 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.981343 12323 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.983208  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.983554  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:04.984030  8889 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:04.984093  8898 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:04.983229  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.969431 11870 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:04.988608  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.989907 12066 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:04.991637  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:04.992249  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:04.993115  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:05.009716  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:05.013043  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.015944 11935 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.016758 12323 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.017385 12323 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:05.017860 12323 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.018900 11935 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:05.019289 11935 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.020026 11935 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.014355  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.021581  8743 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:05.027017  9414 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.547s (SENT)
I20260709 13:01:05.039836  9412 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:05.023694  8818 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 4 to 7, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:05.011853 12228 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.049549  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:05.054384  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:04.957074 12193 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.013015 12246 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:05.056005 12045 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.021096 12262 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.071448 12045 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:05.071946 12045 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.072065  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:05.075724  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61445.1us
I20260709 13:01:05.022909 12066 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:05.082019  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.087185  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:05.091853  8898 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:05.058522 11935 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.057806 12065 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.058815 12246 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:05.030897 12242 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:05.031093 12291 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:01:05.109442  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84872.5us
I20260709 13:01:05.043146 12074 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:01:05.074905 12125 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:05.091521 12055 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:05.045732 12111 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:01:05.048552 11847 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:01:05.120874 11891 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.124257 12065 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.157461 12332 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:05.130190 12246 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:05.157437  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 75538.7us
I20260709 13:01:05.142385 12117 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.149564 12055 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.151230 12242 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:05.214963  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 143016us
I20260709 13:01:05.223675  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:05.202790 12294 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:01:05.187815 12065 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.187829 12332 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:05.226255 12328 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:05.218379 12055 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:05.228369 12325 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:01:05.191476 12233 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:01:05.217088 12242 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:05.229619 12244 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.188125 11891 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.233830 12332 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:05.257788 12320 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 13:01:05.254698 12074 raft_consensus.cc:2764] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:05.203045 12066 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.254094 12125 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.260852 12266 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.261971 12244 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:05.319401  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 95839.5us
I20260709 13:01:05.361159 12125 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.361951  9412 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.363492  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:05.299324  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 109793us
I20260709 13:01:05.365965  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.367395  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:05.367671 12321 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.287s	user 0.001s	sys 0.005s
I20260709 13:01:05.369222  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:05.370496 12074 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.371207 12074 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:05.371675 12074 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:05.372494 12074 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.373548 12111 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.374416 12111 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:05.375773  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.376118  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:05.376401  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.377115  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.404974  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.405012  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:05.405602  9441 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:05.406517  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.409350  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.413961 12340 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:05.414388 12340 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.414532  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.415313 12340 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.416472  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.417161  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.417764  8824 raft_consensus.cc:2376] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10. Re-sending same reply.
I20260709 13:01:05.418293  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.428715  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:05.429582  8749 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.429780  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:05.432260  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.433790  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.265460 11847 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:05.434185  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:05.373728  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:05.434506  8891 raft_consensus.cc:2376] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11. Re-sending same reply.
I20260709 13:01:05.435130  9442 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:05.435432  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.436470  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:05.438066 12237 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:05.438450 12309 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.438625 12237 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:05.439602 12237 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.443135 12244 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.360630 12193 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.276561 12331 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.445575  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:05.378697  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:05.384626  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 542) took 2037 ms (client timeout 1845 ms). Trace:
I20260709 13:01:05.384680 12330 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.385588 12198 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.379437 12266 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.448089 12198 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.448350 12266 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.450286 12198 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:05.450446 12266 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.390389 12232 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.389703  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:05.451303 12232 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:05.451710 12232 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:05.447589  8817 rpcz_store.cc:269] 0709 13:01:03.351402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.354910 (+  3508us) service_pool.cc:224] Handling call
0709 13:01:05.384403 (+2029493us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54000203d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:05.384531 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3040245632}
I20260709 13:01:05.452286 12232 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:05.392107 12042 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:05.399060  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:05.400159  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:05.454563  9448 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:05.401026  8748 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:05.457446  8820 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 5.
W20260709 13:01:05.458889  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66450.5us
I20260709 13:01:05.461911 12198 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:05.462585 12312 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.533s	user 0.006s	sys 0.000s
I20260709 13:01:05.463770 12196 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:05.464270 12196 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:05.462426 12309 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.388809 12278 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Committing config change with OpId 8.7: config changed from index 4 to 7, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:05.433948 12188 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:05.377210 12111 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:05.359575 11891 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:05.450938 12330 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.466565 12188 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.471694 12330 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:05.391106 12195 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.398360  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:05.456961  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.402125  8744 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:01:05.453040 12349 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.093s	user 0.003s	sys 0.000s
I20260709 13:01:05.465257 12196 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.472411 12195 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.475230 12111 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.477207 12195 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:05.489187  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.490909  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.469522  8740 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:05.492570  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.445688 12331 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.500828  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:05.502449  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:05.503705  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.506541  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:05.508873  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.509765  8815 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:05.510576  8740 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:05.515413  8742 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 6: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:05.509505  8824 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:05.319406 12242 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.519809  8822 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:05.522746  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62852.9us
I20260709 13:01:05.497625 12331 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.538949  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.541677  8821 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:05.542677  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.498811 11891 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:01:05.495352 12283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.000s	sys 0.008s
I20260709 13:01:05.544257  8749 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:05.545276  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:05.528198 12326 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:01:05.550906  8822 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.526829 12266 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:05.499286 12225 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:05.552335  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:05.557924  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:05.550388 12283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.000s	sys 0.009s
W20260709 13:01:05.549014 12263 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.002s
I20260709 13:01:05.571796  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:05.572935  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.456902 12362 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.575713 12362 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:05.576195 12362 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:05.578626 12348 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.218s	user 0.003s	sys 0.000s
I20260709 13:01:05.582540  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.586488  8821 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:05.595860  8896 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:05.599516  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:05.600154  8896 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.605625  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.607509 12196 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:05.609498 12329 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
W20260709 13:01:05.610262 12344 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.249s	user 0.003s	sys 0.000s
I20260709 13:01:05.541788 12195 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:01:05.565155 12266 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.571846 12244 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:05.617219  8714 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.019s
I20260709 13:01:05.618453  8896 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:05.604638 12332 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.581205 12262 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:05.574936 12263 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.624702  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.559353 12201 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:05.630649  9448 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:01:05.628129  8714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.000s	sys 0.020s
I20260709 13:01:05.636672  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.637570  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.652040  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:05.627058 12193 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.655709  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.631567 12244 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.671984  8822 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:05.681519  8823 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:05.678997  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.634951 12345 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.684343 12345 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.687292  8815 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:05.688510  8822 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:05.680630  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.640782 12251 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.696446  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:05.643080 12262 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:05.699232  8816 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:05.702816  8821 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:05.709918  8815 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:05.690101 12254 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.715080  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 347835us
I20260709 13:01:05.710402  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:05.700222 12235 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.757776  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:05.762189  9441 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:05.689684 12066 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:01:05.772560  8712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.023s	sys 0.095s
W20260709 13:01:05.773195  8712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.024s	sys 0.095s
W20260709 13:01:05.751328 12246 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.002s
I20260709 13:01:05.691043 12193 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:05.768556  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.764580 12207 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.008s	sys 0.015s
W20260709 13:01:05.774076  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 840687us
I20260709 13:01:05.786358  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:05.789505  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:05.774669  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.017s
I20260709 13:01:05.765874 11891 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:05.791735  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.007s	sys 0.017s
I20260709 13:01:05.797358  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:05.750123 12227 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:01:05.771683 12345 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:05.787701 12207 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.008s	sys 0.015s
I20260709 13:01:05.706146 12251 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:05.802392  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.814785  8819 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:05.811134  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:05.800724  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.017s	sys 0.015s
W20260709 13:01:05.820158  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.017s	sys 0.015s
W20260709 13:01:05.820520  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 689149us
I20260709 13:01:05.791325 12111 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.826911  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.804925 12286 raft_consensus.cc:2764] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:05.800253 12370 raft_consensus.cc:2764] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:05.810722 12291 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:05.844103  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.763371 12254 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.846055  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.848461  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:05.779474 12246 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.826932 12327 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:05.864537  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:05.826889 12336 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.466s	user 0.003s	sys 0.000s
I20260709 13:01:05.766553 12244 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.809743 12362 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:05.777112 12266 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:01:05.906226  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57858.9us
I20260709 13:01:05.907253  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:05.859021 12343 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:05.899632  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:05.847409 12330 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:05.882019  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:05.859385 12298 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.868432 12345 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:05.901383 12045 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.806s	user 0.007s	sys 0.005s
I20260709 13:01:05.839793 12207 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:01:05.819746 12287 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:05.944373  8896 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:05.984404  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.858090 12111 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:05.953087  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.013s	sys 0.011s
I20260709 13:01:05.883601 12327 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.996935  9445 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:05.997392  9445 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:05.810475 12262 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:05.946681  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:06.089368  8897 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.990283  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:05.917126 12291 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.994160  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.013s	sys 0.012s
I20260709 13:01:06.090716  8892 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:05.940173  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58247.2us
I20260709 13:01:05.997041  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:05.905064 12235 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.092726  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:05.864684 12185 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:01:05.952036 12117 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:05.947644 12332 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:05.946098 12195 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.980211 12283 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.989544 12303 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:01:06.154328  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61711.2us
I20260709 13:01:06.155174  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:06.134919  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2197) took 1112 ms. Trace:
I20260709 13:01:06.122012  9445 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.086370 12327 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.156538  9445 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.157930  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 729) took 1941 ms. Trace:
I20260709 13:01:06.125211  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:06.156083  8818 rpcz_store.cc:276] 0709 13:01:05.022941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.023104 (+   163us) service_pool.cc:224] Handling call
0709 13:01:05.023414 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.023588 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:05.492354 (+468766us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.134429 (+642075us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:06.134528 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:06.134588 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.134679 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.134843 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28147,"raft.queue_time_us":187878,"spinlock_wait_cycles":64640,"thread_start_us":613333,"threads_started":2}
I20260709 13:01:06.169548  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:05.947381 12330 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.097560 12066 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:05.976809 12045 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:06.143293 12188 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.000s
I20260709 13:01:06.163955  9446 rpcz_store.cc:276] 0709 13:01:04.197809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.197983 (+   174us) service_pool.cc:224] Handling call
0709 13:01:04.199779 (+  1796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:05.023029 (+823250us) spinlock_profiling.cc:243] Waited 6.47 ms on lock 0x7b1800168e80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64c7bfb 00007f90f64d264b
0709 13:01:05.038789 (+ 15760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:05.038832 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:05.070261 (+ 31429us) log.cc:844] Serialized 52 byte log entry
0709 13:01:05.215823 (+145562us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:05.215875 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.215969 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:05.649267 (+433298us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:06.139073 (+489806us) raft_consensus.cc:1685] finished
0709 13:01:06.139090 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.139220 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14899,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5696,"spinlock_wait_cycles":98091776,"thread_start_us":98079,"threads_started":2,"wal-append.queue_time_us":159953}
I20260709 13:01:06.210873  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:06.202044  8897 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:06.156915 12370 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:06.163199 12201 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:06.262657  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:06.216992 12323 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.199s	user 0.009s	sys 0.005s
I20260709 13:01:05.941887 12343 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.115029 12291 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:06.217211 12232 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.764s	user 0.012s	sys 0.000s
I20260709 13:01:06.198449 12198 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.261961 12371 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:06.203549 12332 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:06.247691 12195 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:06.195843 12372 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.289075  9446 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:06.213671 12193 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:06.284334  9445 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:01:06.249826 12207 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.254412 12357 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:06.318472 12363 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
I20260709 13:01:06.261650 12201 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:06.324199  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 417141us
I20260709 13:01:06.164889 12283 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.339533  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:06.311679 12332 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.312256 12254 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:06.312989 12346 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.459s	user 0.000s	sys 0.001s
I20260709 13:01:06.313653 12297 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:06.313990 12055 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.052s	user 0.011s	sys 0.000s
W20260709 13:01:06.314538 12196 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.707s	user 0.007s	sys 0.002s
I20260709 13:01:06.314641 12237 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.411754  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.413031  8742 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:06.281252 12333 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.516s
I20260709 13:01:06.316815 12372 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:06.317739 12366 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 13:01:06.289870 12252 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.443s	user 0.013s	sys 0.001s
I20260709 13:01:06.309805  9445 pending_rounds.cc:85] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 7
I20260709 13:01:06.422698  9445 pending_rounds.cc:107] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:01:06.423049  9445 raft_consensus.cc:2889] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:06.423839  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 705) took 1049 ms. Trace:
I20260709 13:01:06.291421 12368 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:06.347429 12287 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:06.351158  8818 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:06.352625  8821 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:06.353677  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:06.355968  8743 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:06.299371 12232 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.213007 12066 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.367048  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.433620  8740 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:06.254694 12188 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:06.434538 12228 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.363s	user 0.013s	sys 0.000s
I20260709 13:01:06.435060 12228 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.435961 12188 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.425477 12368 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:06.436539 12383 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:06.411968  8895 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
W20260709 13:01:06.437451 12359 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.536s	user 0.001s	sys 0.001s
I20260709 13:01:06.315455 12253 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.399s
I20260709 13:01:06.439420  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 559) took 1003 ms. Trace:
I20260709 13:01:06.439790  8891 rpcz_store.cc:276] 0709 13:01:05.436145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.437451 (+  1306us) service_pool.cc:224] Handling call
0709 13:01:05.437726 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.438028 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:05.438551 (+   523us) log.cc:844] Serialized 14 byte log entry
0709 13:01:05.629947 (+191396us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:06.360819 (+730872us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:06.378590 (+ 17771us) log.cc:844] Serialized 26 byte log entry
0709 13:01:06.383238 (+  4648us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:06.383289 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.383413 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.439136 (+ 55723us) raft_consensus.cc:1685] finished
0709 13:01:06.439232 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.439367 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":2658,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":2693,"spinlock_wait_cycles":7855488,"thread_start_us":189805,"threads_started":1,"wal-append.queue_time_us":378024}
W20260709 13:01:06.316715 12352 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:01:06.416425 12065 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.169s	user 0.010s	sys 0.001s
I20260709 13:01:06.441972 12330 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:06.442525 12365 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.510s	user 0.000s	sys 0.001s
I20260709 13:01:06.442066 12065 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:06.443068 12369 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.550s	user 0.000s	sys 0.001s
W20260709 13:01:06.444012 12354 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.503s	user 0.000s	sys 0.001s
I20260709 13:01:06.445046  8817 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:06.424301  9445 rpcz_store.cc:276] 0709 13:01:05.374710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.374923 (+   213us) service_pool.cc:224] Handling call
0709 13:01:05.375201 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.996817 (+621616us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400662220. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d11c0 00007f90f64d2231 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:01:06.423546 (+426729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.423762 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862908288}
I20260709 13:01:06.290535 12323 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.400261 12297 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:06.360123  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.340606 12357 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.425220  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:06.450591  8744 raft_consensus.cc:2376] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11. Re-sending same reply.
I20260709 13:01:06.397605 12195 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:01:06.451861 12357 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.451995  9445 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:06.452543  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 558) took 1020 ms. Trace:
I20260709 13:01:06.453617 12045 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:06.437513 12367 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.452s	user 0.000s	sys 0.001s
I20260709 13:01:06.453852  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:06.405429 12251 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:06.413344 12125 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.454564 12111 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.307055 12371 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.455422 12332 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:06.455176 12125 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.456991 12323 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:06.452934  8894 rpcz_store.cc:276] 0709 13:01:05.432150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.434375 (+  2225us) service_pool.cc:224] Handling call
0709 13:01:05.434672 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.459918 (+ 25246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:05.459946 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:05.460395 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:01:05.650126 (+189731us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:05.650188 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.651497 (+  1309us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.452354 (+800857us) raft_consensus.cc:1685] finished
0709 13:01:06.452374 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.452483 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24290,"raft.queue_time_us":26664,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":189329,"threads_started":1,"wal-append.queue_time_us":364369}
I20260709 13:01:06.454617  8821 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:06.471992  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:06.449913  8749 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:06.414459  9442 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:01:06.457208 12371 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:06.447755 12065 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.458691 12125 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:06.462275  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:06.476400 12384 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:06.437237 12383 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.479118 12384 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.479892 12232 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:06.412423 12055 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.379702 12246 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:06.483716  9440 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:06.463151 12297 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:06.487749 12374 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:01:06.487890  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.488660  8749 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:06.473598 12207 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:06.491077 12382 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:01:06.474809 12201 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.485998 12055 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.502522 12201 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:06.505906 12335 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.145s	user 0.008s	sys 0.009s
I20260709 13:01:06.483677 12125 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:06.523186 12343 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.515089 12291 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:06.526582 12066 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:06.483259 12384 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.480567 12383 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.494971 12207 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:06.546823  8749 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:06.513008 12362 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.533829 12283 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:06.557478 11935 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.433s	user 0.008s	sys 0.005s
I20260709 13:01:06.572741  8749 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:06.573578 12333 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.574433  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.587108  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.588188  8740 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:06.574642  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.575387 12340 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.575984  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.589195  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:06.577205  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:06.534981 12125 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.578104  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:06.582293  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.594085  8747 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:06.590008  8744 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:06.576907  8745 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.591017  8746 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:06.596698 12333 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:06.599833 12360 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.035s	user 0.006s	sys 0.007s
I20260709 13:01:06.563851 12383 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:06.571257 12283 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:06.656586  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:06.546983 12207 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:06.584390 12253 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:06.633226 12278 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.166s	user 0.009s	sys 0.004s
I20260709 13:01:06.599889 12327 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.657567  9445 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
W20260709 13:01:07.181968  9412 leader_election.cc:336] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.790s (SENT)
W20260709 13:01:07.452319  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.456017  8816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.008s	sys 0.016s
W20260709 13:01:07.039553  8864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.047s	sys 0.056s
W20260709 13:01:06.952620  9447 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.008s	sys 0.021s
W20260709 13:01:07.291298  9448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.010s	sys 0.013s
W20260709 13:01:07.142167  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.119469  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1557) took 1494 ms. Trace:
W20260709 13:01:07.929836  9447 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.744s	user 0.022s	sys 0.054s
W20260709 13:01:07.458909  8816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.009s	sys 0.016s
I20260709 13:01:07.930995  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 739) took 2121 ms. Trace:
I20260709 13:01:07.931432  9447 rpcz_store.cc:276] 0709 13:01:05.809556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.809708 (+   152us) service_pool.cc:224] Handling call
0709 13:01:05.827343 (+ 17635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.836773 (+  9430us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:05.839475 (+  2702us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.930522 (+2091047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:07.930606 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:07.930670 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.930766 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.930917 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36162,"prepare.queue_time_us":2499,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":157312,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":20873,"thread_start_us":2054267,"threads_started":2,"wal-append.queue_time_us":560980,"wal-append.run_cpu_time_us":1510,"wal-append.run_wall_time_us":1095490}
W20260709 13:01:06.869942 12358 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.461s	user 0.002s	sys 0.000s
I20260709 13:01:06.937899  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:07.934283  8740 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:06.656268 12283 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:07.934948  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 534) took 1358 ms (client timeout 1790 ms). Trace:
W20260709 13:01:07.935436  8740 rpcz_store.cc:269] 0709 13:01:06.576479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.592229 (+ 15750us) service_pool.cc:224] Handling call
0709 13:01:07.934893 (+1342664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944}
I20260709 13:01:07.936723  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:07.937052 12283 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:07.741775 12255 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.015s	sys 0.045s
W20260709 13:01:07.758175 12297 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.270s	user 0.012s	sys 0.004s
I20260709 13:01:06.634603 11935 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:07.938567 12297 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:07.938935 11935 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:06.657793  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:07.118856  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1546) took 1626 ms. Trace:
I20260709 13:01:07.941514 11935 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:06.945946  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.004s	sys 0.025s
W20260709 13:01:07.168061 12198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.004s	sys 0.031s
W20260709 13:01:07.942695  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.755s	user 0.013s	sys 0.056s
W20260709 13:01:07.459055  8864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.055s	sys 0.068s
W20260709 13:01:07.455224 12300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.022s	sys 0.024s
W20260709 13:01:07.076453  8889 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.004s	sys 0.024s
W20260709 13:01:07.943888 12300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.804s	user 0.024s	sys 0.031s
W20260709 13:01:07.944252  8889 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.672s	user 0.015s	sys 0.048s
W20260709 13:01:07.943089  9448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.520s	user 0.026s	sys 0.019s
W20260709 13:01:06.758081 12227 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.007s	sys 0.014s
W20260709 13:01:07.124964 12266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.000s	sys 0.020s
W20260709 13:01:07.727679 12409 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:01:07.945273 12227 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.783s	user 0.023s	sys 0.055s
I20260709 13:01:07.859826 12397 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:07.490934 12253 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:07.946493 12397 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.947214  8824 raft_consensus.cc:2410] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:06.933537  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.948093  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 710) took 1334 ms (client timeout 1609 ms). Trace:
I20260709 13:01:07.948750 12397 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:07.083710  8890 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.012s	sys 0.015s
W20260709 13:01:07.281396 11891 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.157s	user 0.005s	sys 0.018s
W20260709 13:01:07.949257  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.937673  8823 rpcz_store.cc:276] 0709 13:01:05.624865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.638462 (+ 13597us) service_pool.cc:224] Handling call
0709 13:01:05.681186 (+ 42724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.681931 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.119249 (+1437318us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b540028bc50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:07.119405 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34288000}
W20260709 13:01:07.949898  8890 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.806s	user 0.012s	sys 0.048s
W20260709 13:01:07.458455  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 276602us
W20260709 13:01:06.732167  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.018s	sys 0.012s
I20260709 13:01:06.931583  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:07.932562 12327 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:06.732334  9449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.015s	sys 0.028s
W20260709 13:01:07.136128  8862 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.023s	sys 0.020s
W20260709 13:01:07.455425  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.943014  8741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.253s	user 0.013s	sys 0.018s
W20260709 13:01:06.656584 12331 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.106s	user 0.004s	sys 0.009s
W20260709 13:01:07.362851 12286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.006s	sys 0.000s
W20260709 13:01:07.937778 12255 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.440s	user 0.018s	sys 0.050s
I20260709 13:01:07.544691 12263 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.504944 12403 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:07.953439  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 548) took 1100 ms. Trace:
W20260709 13:01:07.948822  9443 rpcz_store.cc:269] 0709 13:01:06.613449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.657444 (+ 43995us) service_pool.cc:224] Handling call
0709 13:01:07.948025 (+1290581us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:06.705279 12385 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:07.957218  9442 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:07.957985  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:07.455732 12117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.008s	sys 0.020s
I20260709 13:01:07.544886  8821 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:07.506697 12340 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.907s	user 0.012s	sys 0.001s
W20260709 13:01:07.115937 12042 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:01:06.650799 12383 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:06.959729 12353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.025s
W20260709 13:01:07.476104 12298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.004s	sys 0.023s
I20260709 13:01:07.991027 12340 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:07.991312 12298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.837s	user 0.008s	sys 0.028s
W20260709 13:01:07.952244  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 1.70172e+06us
W20260709 13:01:07.959600  9449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.800s	user 0.031s	sys 0.064s
W20260709 13:01:07.960181  9446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.010s
I20260709 13:01:07.991657 12340 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.992060  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:07.992297  9446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.007s	sys 0.010s
I20260709 13:01:07.993121  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 666) took 1686 ms. Trace:
I20260709 13:01:07.993489  9446 rpcz_store.cc:276] 0709 13:01:06.306512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.317751 (+ 11239us) service_pool.cc:224] Handling call
0709 13:01:06.318119 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.318332 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:06.404074 (+ 85742us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.114111 (+710037us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003aebe0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:01:07.992682 (+878571us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.992832 (+   150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:07.992874 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:07.992916 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.992972 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.993071 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97964,"prepare.queue_time_us":651885,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3274,"raft.queue_time_us":70711,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":31149,"spinlock_wait_cycles":168146688,"thread_start_us":825949,"threads_started":2,"wal-append.queue_time_us":1337310}
I20260709 13:01:07.995426  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:07.995926  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:07.996328  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.996867  9446 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:07.544328 12406 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:07.960402  8741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.271s	user 0.013s	sys 0.018s
I20260709 13:01:07.997479  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 738) took 2216 ms. Trace:
I20260709 13:01:07.997589 12406 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:07.998042  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 531) took 1422 ms. Trace:
I20260709 13:01:07.998564 12392 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:07.943113 12198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.603s	user 0.020s	sys 0.059s
I20260709 13:01:07.999207 12392 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.999500 12198 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:07.999723 12392 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.000576 12198 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.941162 12297 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:07.967211  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 707) took 2546 ms. Trace:
I20260709 13:01:07.998632  8741 rpcz_store.cc:276] 0709 13:01:06.575553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.577815 (+  2262us) service_pool.cc:224] Handling call
0709 13:01:06.584041 (+  6226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:06.584562 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:06.584608 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:06.602145 (+ 17537us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.997695 (+1395550us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:07.997740 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.997811 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:07.997858 (+    47us) raft_consensus.cc:1685] finished
0709 13:01:07.997871 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.997988 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36901,"raft.queue_time_us":128,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3841,"thread_start_us":1308087,"threads_started":1,"wal-append.queue_time_us":1307708}
I20260709 13:01:07.985953 12283 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:07.959841  8862 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.603s	user 0.029s	sys 0.037s
I20260709 13:01:07.987154 12385 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.006155 12406 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:08.008939 12406 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:07.979761 12263 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.240726 12396 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.931102 12362 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:07.945420 12266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.020s	sys 0.033s
W20260709 13:01:07.961457  9414 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.004s	sys 0.013s
I20260709 13:01:07.960889 12405 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:07.961390 12397 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:07.971060 12361 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.512s	user 0.000s	sys 0.014s
I20260709 13:01:07.968184  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2203) took 1616 ms. Trace:
W20260709 13:01:07.958583 11891 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.843s	user 0.020s	sys 0.025s
I20260709 13:01:07.443486 12189 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.941591  8819 rpcz_store.cc:276] 0709 13:01:05.492250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.494859 (+  2609us) service_pool.cc:224] Handling call
0709 13:01:05.504670 (+  9811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.815214 (+310544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.118518 (+1303304us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54003e2bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:07.118775 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712929536}
I20260709 13:01:07.543462 12394 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:07.535837 12125 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:06.932130 12333 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:07.977808  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:07.965391 12286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.820s	user 0.008s	sys 0.012s
I20260709 13:01:07.679374 12408 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:07.956915  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.770s	user 0.031s	sys 0.058s
I20260709 13:01:08.063392 12423 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:08.065209  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 559) took 1625 ms (client timeout 2091 ms). Trace:
I20260709 13:01:08.043502 12405 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:08.065645  8823 rpcz_store.cc:269] 0709 13:01:06.439427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.961065 (+1521638us) service_pool.cc:224] Handling call
0709 13:01:08.065143 (+104078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.066893  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.067601  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 564) took 1121 ms. Trace:
I20260709 13:01:08.067891  8823 rpcz_store.cc:276] 0709 13:01:06.946906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.066614 (+1119708us) service_pool.cc:224] Handling call
0709 13:01:08.067552 (+   938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.069789  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.071218  8819 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:07.979959  8740 rpcz_store.cc:276] 0709 13:01:06.853334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.936353 (+1083019us) service_pool.cc:224] Handling call
0709 13:01:07.953383 (+ 17030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.981076  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 708) took 1371 ms. Trace:
I20260709 13:01:07.981573  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 712) took 1051 ms. Trace:
I20260709 13:01:08.072628  9447 rpcz_store.cc:276] 0709 13:01:06.930190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.959643 (+1029453us) service_pool.cc:224] Handling call
0709 13:01:07.979629 (+ 19986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.979809 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.979835 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:07.979876 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:07.979906 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.979955 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.981502 (+  1547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.958763 12399 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.070410 12408 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.073959 12399 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.075893 12408 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.076069 12399 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:07.958750 12398 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:08.078317 12398 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.958859 12390 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.081678 12390 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.082629 12263 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.084160 12390 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:07.968120 12262 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:08.084975 12262 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.987635 12402 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.988595 11935 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.086213 12402 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.086323 11935 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:07.989081 12386 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:08.088656 12402 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:07.989562  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:08.089457 12266 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.089983  9443 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:08.089087  8788 connection.cc:256] RPC call timeout handler was delayed by -1.2497218580000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 13:01:08.090161 12398 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.091039 12125 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:08.090029 12266 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.091756 12125 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:07.986685 12233 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.989979 12383 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.093712 12383 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:08.093598 12233 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.094283 12383 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.096647 12383 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:07.980361 12403 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.096830 11592 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:08.097550 12340 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.053906 12189 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.098723 12403 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:07.990826 12353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.847s	user 0.016s	sys 0.055s
W20260709 13:01:07.982507 12117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.819s	user 0.015s	sys 0.022s
I20260709 13:01:08.006155  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:07.997898  9448 rpcz_store.cc:276] 0709 13:01:05.780741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.780866 (+   125us) service_pool.cc:224] Handling call
0709 13:01:05.801596 (+ 20730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:06.169774 (+368178us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:06.340837 (+171063us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.945220 (+1604383us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.962522 (+ 17302us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.962691 (+   169us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:07.962714 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.962760 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:07.997241 (+ 34481us) raft_consensus.cc:1685] finished
0709 13:01:07.997260 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.997424 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108619,"prepare.queue_time_us":160067,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":177954,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":12834,"thread_start_us":1855037,"threads_started":2,"wal-append.queue_time_us":1254003}
I20260709 13:01:08.102566  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:08.103474  8741 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:08.048722 12397 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.006709 12283 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.004101  9441 rpcz_store.cc:276] 0709 13:01:05.421155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.421331 (+   176us) service_pool.cc:224] Handling call
0709 13:01:05.421614 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.763964 (+342350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.966957 (+2202993us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:07.967146 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760886528}
I20260709 13:01:08.005244 12244 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.103272 12117 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:08.106699 12117 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.107833 12283 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.042268 12362 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.044606  8816 rpcz_store.cc:276] 0709 13:01:06.351786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.355274 (+  3488us) service_pool.cc:224] Handling call
0709 13:01:06.355621 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:06.355972 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:06.356001 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:06.356525 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.930878 (+1574353us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:07.964675 (+ 33797us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.967775 (+  3100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.967883 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:07.967930 (+    47us) raft_consensus.cc:1685] finished
0709 13:01:07.967946 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.968113 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11235,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":82652,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":157,"thread_start_us":1607415,"threads_started":2,"wal-append.queue_time_us":1293516}
I20260709 13:01:08.109763 12362 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:08.110188  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.110355 12189 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.110689  8741 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:08.110805  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:08.056136 12394 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:08.111235  8741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 549) took 1257 ms (client timeout 1602 ms). Trace:
I20260709 13:01:08.111353  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 565) took 1116 ms. Trace:
I20260709 13:01:07.988071 12042 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:08.111654  8741 rpcz_store.cc:269] 0709 13:01:06.853648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.005876 (+1152228us) service_pool.cc:224] Handling call
0709 13:01:08.111177 (+105301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.112159 12392 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.112702 12394 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.112823  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:08.114440 12392 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:08.111759  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.115447  8823 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:08.115914  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 563) took 1465 ms. Trace:
I20260709 13:01:08.116202  8823 rpcz_store.cc:276] 0709 13:01:06.650584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.111509 (+1460925us) service_pool.cc:224] Handling call
0709 13:01:08.115857 (+  4348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.117100  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.120894  8741 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.111719  8816 rpcz_store.cc:276] 0709 13:01:06.994721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.109942 (+1115221us) service_pool.cc:224] Handling call
0709 13:01:08.111295 (+  1353us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:08.050971  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43491.6us
I20260709 13:01:07.977413 12331 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.009555 12385 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.059525 12424 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:08.091147 12408 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:07.989871  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:08.097714 11592 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:08.009491 12406 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:07.987603  8787 connection.cc:256] RPC call timeout handler was delayed by -1.8194986390000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 13:01:08.096676  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:08.100714  9442 rpcz_store.cc:276] 0709 13:01:06.609479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.609668 (+   189us) service_pool.cc:224] Handling call
0709 13:01:07.981015 (+1371347us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:08.036245  9414 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.005s	sys 0.013s
I20260709 13:01:08.040328 12396 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.055692 12198 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:08.068446 12405 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:08.241052  8820 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:08.245954  8748 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:08.259567  8819 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:01:08.274822  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 872237us
I20260709 13:01:08.163823  8740 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:08.248132  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.177994  9444 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:08.153944  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 165650us
I20260709 13:01:08.260105  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.276732  8745 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12. Re-sending same reply.
I20260709 13:01:08.276036  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 529) took 1701 ms. Trace:
I20260709 13:01:08.154663  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:08.276571  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 179994us
I20260709 13:01:08.278115  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:08.279151  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 540) took 1438 ms (client timeout 1515 ms). Trace:
I20260709 13:01:08.185096  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:08.153635 12189 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:08.280969  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 545) took 1428 ms (client timeout 1756 ms). Trace:
I20260709 13:01:08.281617  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:08.282207  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:08.287146  8741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 544) took 1436 ms (client timeout 1773 ms). Trace:
I20260709 13:01:08.280201  8816 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.278596  8748 rpcz_store.cc:276] 0709 13:01:06.574944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.577605 (+  2661us) service_pool.cc:224] Handling call
0709 13:01:08.245453 (+1667848us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:08.245691 (+   238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:08.275759 (+ 30068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.275980 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099176704}
I20260709 13:01:08.356225  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1560) took 2730 ms. Trace:
I20260709 13:01:08.517591  8815 rpcz_store.cc:276] 0709 13:01:05.625702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.691519 (+ 65817us) service_pool.cc:224] Handling call
0709 13:01:05.709618 (+ 18099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:05.714112 (+  4494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.355936 (+2641824us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b54003ec1d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:08.356149 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15768832}
I20260709 13:01:08.419992  9446 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:08.468322  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.361541  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 84291.6us
I20260709 13:01:08.521335  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:08.508376  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.522707  8740 rpcz_store.cc:269] 0709 13:01:06.852322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.073314 (+1220992us) service_pool.cc:224] Handling call
0709 13:01:08.280907 (+207593us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:08.361578  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 81442.6us
I20260709 13:01:08.447629  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:08.526101  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.529858  8741 rpcz_store.cc:269] 0709 13:01:06.851183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.112598 (+1261415us) service_pool.cc:224] Handling call
0709 13:01:08.287084 (+174486us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:08.394994  8712 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.540s (SENT) [suppressed 20 similar messages]
W20260709 13:01:08.497840  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70863.7us
W20260709 13:01:08.462945  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 505092us
I20260709 13:01:08.463027  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.409335  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 129121us
W20260709 13:01:08.532481  9415 connection.cc:256] RPC call timeout handler was delayed by -1.0327782429999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
I20260709 13:01:08.530469  9448 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
W20260709 13:01:08.287976  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 240126us
I20260709 13:01:08.257493 12142 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.145571 11592 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:08.357769  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:08.536485  9414 connection.cc:256] RPC call timeout handler was delayed by -1.4075828769999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
W20260709 13:01:08.537089  9414 connection.cc:256] RPC call timeout handler was delayed by -1.3868828769999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 13:01:08.537257  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.537484  9414 connection.cc:256] RPC call timeout handler was delayed by -1.2932828769999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:01:08.537998  9414 connection.cc:256] RPC call timeout handler was delayed by -1.2365828769999325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:01:08.538688  9414 connection.cc:256] RPC call timeout handler was delayed by -1.1669610699999566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:01:08.539047  9414 connection.cc:256] RPC call timeout handler was delayed by -1.1247888489999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:01:08.176491 12288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.817s	user 0.009s	sys 0.039s
I20260709 13:01:08.538981  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.529307  8745 rpcz_store.cc:269] 0709 13:01:06.840423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.840561 (+   138us) service_pool.cc:224] Handling call
0709 13:01:08.248010 (+1407449us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:08.279097 (+ 31087us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091289088}
I20260709 13:01:08.182101 12385 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:08.549219  9443 raft_consensus.cc:2368] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:08.552662  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:08.153580 12392 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:08.357968 12403 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.563953  8816 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:08.564669  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.277885 12195 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.281243 12394 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:08.549804 12288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.191s	user 0.023s	sys 0.063s
I20260709 13:01:08.355657 12386 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:08.282007 12314 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:08.287246 12283 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.278162 12399 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:08.280848 12049 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:08.280956 12235 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:08.563490 12345 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:08.410847 12286 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:08.568398  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.614460  8748 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:08.615094  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 542) took 1765 ms (client timeout 2059 ms). Trace:
W20260709 13:01:08.615525  8748 rpcz_store.cc:269] 0709 13:01:06.849639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.547365 (+1697726us) service_pool.cc:224] Handling call
0709 13:01:08.615019 (+ 67654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.555831  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:08.617115  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61396.4us
I20260709 13:01:08.571451  8815 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:08.145566 12424 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.557312 12385 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.540020 12362 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:08.541778 12390 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.533458 12422 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:08.181869 12408 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.587502 12435 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:08.587792  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1544) took 3097 ms (client timeout 1532 ms). Trace:
I20260709 13:01:08.410713 11891 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.157s
I20260709 13:01:08.582515  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.420789 12402 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.157s
I20260709 13:01:08.285578 12405 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:08.281390 12198 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.540570 12189 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:08.613468  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78263us
W20260709 13:01:08.622305  8864 connection.cc:256] RPC call timeout handler was delayed by -1.8859962829999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:01:08.584450 12255 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:08.579741 12392 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.673915  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:08.664687  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 136724us
I20260709 13:01:08.675973  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:08.678136  8864 connection.cc:256] RPC call timeout handler was delayed by -1.8040962830000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:01:08.610682 12049 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.582484 12394 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.664882  8816 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:08.693246  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.693440 11870 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.693599  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.694432  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.695809  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.696880  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:08.697100  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.698480  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.699335  8816 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:08.700248  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.700730 12262 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:08.659385  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 127266us
I20260709 13:01:08.613744 12288 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:08.660893 12042 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.002s
W20260709 13:01:08.687124  8864 connection.cc:256] RPC call timeout handler was delayed by -1.3621962830000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 13:01:08.656459 12385 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:08.663941  8817 rpcz_store.cc:269] 0709 13:01:05.490572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.490689 (+   117us) service_pool.cc:224] Handling call
0709 13:01:08.587551 (+3096862us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b5400082fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:08.587695 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2193420032}
I20260709 13:01:08.581310 12195 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.583894 12403 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:08.691959  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 543) took 1839 ms (client timeout 2082 ms). Trace:
I20260709 13:01:08.740278  8891 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:08.613837 11592 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:08.740463  8748 rpcz_store.cc:269] 0709 13:01:06.849978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.616453 (+1766475us) service_pool.cc:224] Handling call
0709 13:01:08.689650 (+ 73197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.610610 12428 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.701547  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:08.742794  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 208523us
I20260709 13:01:08.744496  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.673413 12422 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.703538 11870 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:08.704891  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 740) took 2797 ms. Trace:
I20260709 13:01:08.664831 12434 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.610414 12399 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.708369  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:08.760308  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 533) took 2184 ms (client timeout 1509 ms). Trace:
I20260709 13:01:08.679494 12435 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:08.720396  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 325509us
W20260709 13:01:08.760794  8749 rpcz_store.cc:269] 0709 13:01:06.576183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.579579 (+  3396us) service_pool.cc:224] Handling call
0709 13:01:08.708190 (+2128611us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:08.760202 (+ 52012us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087634176}
I20260709 13:01:08.584141 12386 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.619418 12235 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:08.730966  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 113123us
I20260709 13:01:08.740789 12042 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:08.732679  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:08.786156  8895 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.691749 12049 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.554944 12353 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.160s
I20260709 13:01:08.660573 12286 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.735512  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:08.739594  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.708165 12331 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.794852  8817 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:08.673977 12362 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.733400  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:08.807786  9442 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:08.702630 11935 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.687372 12425 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:08.747043  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.787447 11592 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.683490 12405 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.752228  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:08.673630 12390 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.715737 12396 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.755200  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.758479  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.608549 12283 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.753010  9449 rpcz_store.cc:276] 0709 13:01:05.907646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.907778 (+   132us) service_pool.cc:224] Handling call
0709 13:01:05.945865 (+ 38087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:06.138592 (+192727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:06.138635 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:06.159389 (+ 20754us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.992004 (+1832615us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:07.992045 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.704396 (+712351us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001937d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:08.704468 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.704534 (+    66us) raft_consensus.cc:1685] finished
0709 13:01:08.704551 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.704816 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17918,"raft.queue_time_us":78913,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10852,"spinlock_wait_cycles":442444672,"thread_start_us":1832280,"threads_started":1,"wal-append.queue_time_us":1248543,"wal-append.run_cpu_time_us":1811,"wal-append.run_wall_time_us":1018884}
I20260709 13:01:08.766927  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.770334 12436 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:08.771464 12402 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:01:08.773648  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45875.5us
I20260709 13:01:08.780611  8742 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:08.783360  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:08.732231  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:08.737792 12125 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.750114 12397 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.790287  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:08.752584 12406 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.794852  8897 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:08.739665 12288 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.806061  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.806957  8823 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:08.808038  8896 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.817412  8816 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:08.594602 12314 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:08.765658 12434 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.752157 12428 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:08.942166  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 198109us
W20260709 13:01:08.817929  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 139213us
W20260709 13:01:08.979264  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 196046us
I20260709 13:01:08.977843  8748 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:08.980566  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:08.981112  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:08.982601  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:08.982956  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:08.983448  8894 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:08.983016  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 286266us
I20260709 13:01:08.977253  8817 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
W20260709 13:01:08.989017  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 547) took 2136 ms (client timeout 2090 ms). Trace:
I20260709 13:01:08.978403  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:08.989460  8748 rpcz_store.cc:269] 0709 13:01:06.852966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.741275 (+1888309us) service_pool.cc:224] Handling call
0709 13:01:08.988952 (+247677us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.989279  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.789415 12433 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.985822  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.990585  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:08.987782  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:08.995787  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.996434  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.996425  8816 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:08.998310  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:08.998987  8816 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12. Re-sending same reply.
I20260709 13:01:08.999781  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.001618  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.002168  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.003134  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.843146 12444 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:09.001236 11390 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:09.009950 12444 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:09.009506 12396 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:09.010474 12444 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.011088 12396 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:09.011371  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:08.861774 12441 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.012049  8747 raft_consensus.cc:2376] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
W20260709 13:01:09.012594  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 538) took 2361 ms (client timeout 2149 ms). Trace:
W20260709 13:01:08.838914 12426 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 13:01:08.842681 12429 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:01:08.823961 12042 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.192s
I20260709 13:01:09.016470  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.016964 12423 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.012207 12441 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.011549 12396 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:09.018728  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:08.848449 12427 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:01:09.017128  9449 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:09.019280  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:09.020648  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:08.837224 12431 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:01:09.010268 11390 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.070827  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.071708  9443 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:09.012997  8747 rpcz_store.cc:269] 0709 13:01:06.651001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.651150 (+   149us) service_pool.cc:224] Handling call
0709 13:01:09.011201 (+2360051us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:09.012525 (+  1324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105616128}
I20260709 13:01:09.017643 12435 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.022912 12442 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:08.854411 12422 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.836831 12430 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:09.038329 12441 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.014590 12242 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.082541 12386 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.851167 12286 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.081923 12430 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:09.083635 12408 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.039251 12396 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.040762 12207 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.002418 12397 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:09.041492  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.024787  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
W20260709 13:01:09.042593  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1549) took 3523 ms (client timeout 2114 ms). Trace:
I20260709 13:01:09.025009  8817 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:09.026216  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43689.2us
I20260709 13:01:09.036698 12423 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.038002 12195 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.039269 12403 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.044291  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.045377 12443 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:09.026957  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:09.047206 12314 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.047472 12390 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:08.988801 12327 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.048265 12266 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.049638  8744 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:09.049934 12399 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:09.051468  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 558) took 2612 ms (client timeout 1924 ms). Trace:
I20260709 13:01:09.023542  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:08.989482 12437 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:09.053785 12398 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:09.055053 12189 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.055361 12405 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.994732 12425 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.058264 12235 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:08.978333 12424 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.006073 12433 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:08.842794 12394 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.038208 12283 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:09.065263 12244 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.959s	user 0.006s	sys 0.003s
I20260709 13:01:09.068099  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.361696  8892 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.998252 12392 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.090839  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:09.139816  8896 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:09.241922 12408 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.242256  9443 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 8 LEARNER]: Advancing to term 9
W20260709 13:01:09.097074  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 354830us
I20260709 13:01:09.341413  8818 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:09.342710  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.006s	sys 0.021s
W20260709 13:01:09.369890  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.006s	sys 0.021s
I20260709 13:01:09.323273  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.370791  9441 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:09.370556  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.184032  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.313725  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.371807  8818 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:09.202675  8890 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.022s	sys 0.054s
I20260709 13:01:09.346375  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 711) took 2720 ms. Trace:
I20260709 13:01:09.102454  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.344129  8894 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:09.376945  9445 rpcz_store.cc:276] 0709 13:01:06.625969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.503670 (+877701us) service_pool.cc:224] Handling call
0709 13:01:07.965053 (+461383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:08.110400 (+145347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.110437 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.110487 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:08.110527 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.346119 (+1235592us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540065e854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:09.346184 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.346311 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333887872}
I20260709 13:01:09.377811  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.378896  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:09.339744  8821 rpcz_store.cc:269] 0709 13:01:06.439183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.471620 (+ 32437us) service_pool.cc:224] Handling call
0709 13:01:09.051261 (+2579641us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:09.051391 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466412288}
I20260709 13:01:09.379678  8894 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:01:09.084841 12430 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:09.346479  8824 rpcz_store.cc:269] 0709 13:01:05.518910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.519030 (+   120us) service_pool.cc:224] Handling call
0709 13:01:09.042389 (+3523359us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b540012c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:09.042523 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5529185152}
I20260709 13:01:09.352478 12398 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:09.351236  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:09.344189 12347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.036s	user 0.015s	sys 0.039s
I20260709 13:01:09.357939  8788 leader_election.cc:400] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:09.417912  8788 leader_election.cc:403] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:09.359508  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.419178  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:09.419759  8741 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:09.348017 12309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.024s
I20260709 13:01:09.423456  8822 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:09.317031 12396 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.183214 12442 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:09.432025  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 74237.9us
I20260709 13:01:09.365803  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:09.342188  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.015s	sys 0.019s
I20260709 13:01:09.344298 12397 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.077037  8895 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:09.436064  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.353466 12327 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:09.344107  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 317306us
I20260709 13:01:09.345707  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:09.355322 12266 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.381829  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.382401  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.383749  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:09.384529  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.385119  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:09.385676 12451 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:09.385998  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:09.357035 12390 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:09.090075 12444 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.381594  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:09.459594  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73692.3us
W20260709 13:01:09.460244  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78731us
I20260709 13:01:09.410801 12443 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:09.394062 12386 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.394799 12283 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.378801 12408 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.395474 12394 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.396257 12403 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.396895 12314 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.397897 12196 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:09.399063  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 521) took 2976 ms (client timeout 1686 ms). Trace:
W20260709 13:01:09.472000  8742 rpcz_store.cc:269] 0709 13:01:06.426671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.426867 (+   196us) service_pool.cc:224] Handling call
0709 13:01:09.398872 (+2972005us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400464bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:09.398984 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5130840448}
I20260709 13:01:09.473486  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.373781 11592 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.351583 12448 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:09.403905  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 550) took 1124 ms. Trace:
I20260709 13:01:09.079269 11390 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.410458  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 662) took 4024 ms. Trace:
I20260709 13:01:09.409678  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:09.477421  9440 rpcz_store.cc:276] 0709 13:01:05.386278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.402298 (+ 16020us) service_pool.cc:224] Handling call
0709 13:01:05.402576 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.487427 (+1084851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.410212 (+2922785us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:09.410399 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486566144}
W20260709 13:01:09.477919  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 68342.9us
I20260709 13:01:09.421972  8821 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:09.422488  9443 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 9 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:09.433029  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.015s	sys 0.019s
I20260709 13:01:09.434114  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:09.480023  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 560) took 2856 ms. Trace:
W20260709 13:01:09.480211  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46173.9us
I20260709 13:01:09.480496  8821 rpcz_store.cc:276] 0709 13:01:06.624010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.416009 (+2791999us) service_pool.cc:224] Handling call
0709 13:01:09.421740 (+  5731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.479717 (+ 57977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.479959 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.482653 12185 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.482930 12433 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.483088 12185 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.483501 12185 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.484393 12368 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.485514 12368 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.485451 11592 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.434314  8892 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:09.487386  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.488063  8892 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:09.488734  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.436378  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:09.489367  8892 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:09.376641  8890 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.026s	sys 0.059s
I20260709 13:01:09.436771  8895 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:09.463122 12244 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.437868 12402 consensus_queue.cc:1243] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:09.441383  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:09.379962 12449 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.442587  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:09.423889 12309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.000s	sys 0.025s
W20260709 13:01:09.417414 12347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.120s	user 0.016s	sys 0.039s
I20260709 13:01:09.445936  9447 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:09.445627  9445 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:09.447407  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:09.489631  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53342.2us
I20260709 13:01:09.456002  8741 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:09.509949  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1890) took 1104 ms. Trace:
I20260709 13:01:09.510154  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:09.510326  8741 rpcz_store.cc:276] 0709 13:01:08.405108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.418912 (+1013804us) service_pool.cc:224] Handling call
0709 13:01:09.509883 (+ 90971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.456588  8817 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:09.458374  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.461141  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:09.511925  8824 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
W20260709 13:01:09.512077  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51062.7us
I20260709 13:01:09.512903  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:09.423166 12362 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.431468 12393 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.528353 12362 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:09.528504 12393 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.529018 12362 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.442124 12397 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.464488  9441 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:09.410422 12392 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:09.464772 12331 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.534614 12331 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:09.534746  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:09.535597  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:09.536352  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:09.475726  8747 rpcz_store.cc:276] 0709 13:01:08.279184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.073806 (+794622us) service_pool.cc:224] Handling call
0709 13:01:09.403856 (+330050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.535849 12331 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.442970 12456 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:09.537366 12428 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.361227 12437 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.538502  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.465678 12189 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:09.444366  8894 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:09.475304 12448 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:09.511960  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:09.511752  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 70486.2us
I20260709 13:01:09.637095  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.649524  8740 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:09.525784  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1923) took 3581 ms. Trace:
I20260709 13:01:09.471899 12196 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:09.507334 12446 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.534971 12392 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.536784  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.526304 12402 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:09.540973 12433 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:09.490623  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.506085  8896 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:09.689900  8895 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.690653  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.691545  8896 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:09.692607  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.544483  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:09.563887  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.694046  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1894) took 1287 ms. Trace:
I20260709 13:01:09.486228 12235 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:09.635377  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.513616  8818 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:09.522799  8891 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
I20260709 13:01:09.490408 12434 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.642256 12428 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.472152 12403 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.655205  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.474295 12314 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:09.539721 12385 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.002s
I20260709 13:01:09.669483 12446 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.662415  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:09.500586 12398 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.689437 12433 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:09.667266  8713 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.022s	sys 0.041s
I20260709 13:01:09.667948  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:09.749505  8713 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.023s	sys 0.041s
W20260709 13:01:09.750757  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.459103 12266 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.540550 12195 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.469578 12386 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:09.470877 12443 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.472922 12394 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.475685 12288 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.636734 12362 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.499940  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.694538  8747 rpcz_store.cc:276] 0709 13:01:08.406414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.537197 (+1130783us) service_pool.cc:224] Handling call
0709 13:01:09.693971 (+156774us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:09.694259  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.789697  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.634481 12189 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:09.696430  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.640255 12442 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.698025  8891 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:09.638911 12439 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.471958 12283 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.641268 12448 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.708339  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.713276  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.729884 12196 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.514094 12449 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.671448 12402 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.816948 12402 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:09.700398 12426 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:01:09.748606  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 403032us
I20260709 13:01:09.824606  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:09.668901  8821 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:09.650102  8890 rpcz_store.cc:276] 0709 13:01:05.944581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.945471 (+   890us) service_pool.cc:224] Handling call
0709 13:01:05.945807 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:07.962863 (+2017056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.962881 (+    18us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:07.963246 (+   365us) log.cc:844] Serialized 52 byte log entry
0709 13:01:08.176960 (+213714us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:09.513386 (+1336426us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.525100 (+ 11714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.525229 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.525507 (+   278us) raft_consensus.cc:1685] finished
0709 13:01:09.525527 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.525714 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155980,"prepare.queue_time_us":1271582,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":202320,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":21176,"thread_start_us":3136438,"threads_started":3,"wal-append.queue_time_us":609327}
I20260709 13:01:09.754256  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.826187  8740 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:09.826263  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.826825 12460 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.826965  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.827674  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.828052  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.828362  8821 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:09.828672  8740 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
W20260709 13:01:09.828909  9414 connection.cc:256] RPC call timeout handler was delayed by -1.485222752000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260709 13:01:09.829452  9414 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.853s (SENT)
I20260709 13:01:09.829525  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.829856  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:09.826346  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.830601  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.831526  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.702493 12235 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:09.758519  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 96219.1us
I20260709 13:01:09.762228  8895 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:09.834493  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.736053 12428 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:09.791538  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.838042  8892 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:09.838845  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.795199 12189 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.757751 12433 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.746634 12437 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:09.735559 12446 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.794209  8824 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:09.797201  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.844091  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.798575 12406 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:09.845118  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.798259  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 568) took 1064 ms. Trace:
I20260709 13:01:09.763600 12392 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.763760 12423 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:09.786361 12049 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.983s	user 0.009s	sys 0.002s
I20260709 13:01:09.803634  8817 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:09.762039 12195 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:09.809006  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.837169 12235 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.739042 12385 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.831526  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:09.832801  9414 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.149s (SENT)
I20260709 13:01:09.835237  8895 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:09.839787  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.697556  8894 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:09.859411  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.841166  8896 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:09.860194  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.861032  8894 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:09.842159  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.861908  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.862217  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:09.862466  8894 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:09.863219  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.863327  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.863709  8894 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 6.
I20260709 13:01:09.864275  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.864416  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.864872  8894 raft_consensus.cc:2376] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
W20260709 13:01:09.833401 12455 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 13:01:09.860196  8740 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:09.859411  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.865272  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.866328  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.866765  8818 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:09.866870  8740 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:09.867604  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:09.867941 12242 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.786s	user 0.008s	sys 0.002s
I20260709 13:01:09.867995  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.855835  8747 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:09.816339 12449 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:09.856730  8863 leader_election.cc:336] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.752s (SENT)
I20260709 13:01:09.869726  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.870110  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:09.870414  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 546) took 3017 ms. Trace:
I20260709 13:01:09.870474 12385 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.854095  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:09.831817 12390 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.860925  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.845870  8891 rpcz_store.cc:276] 0709 13:01:08.733732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.761688 (+ 27956us) service_pool.cc:224] Handling call
0709 13:01:09.798100 (+1036412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.868193  8818 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:09.871999  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.872541  8746 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:09.872759  8891 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:09.873265  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 572) took 1138 ms. Trace:
I20260709 13:01:09.873219 12423 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.873816  8891 rpcz_store.cc:276] 0709 13:01:08.734652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.871769 (+1137117us) service_pool.cc:224] Handling call
0709 13:01:09.873214 (+  1445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.874377 12423 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:09.874830 12423 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:09.875226 12423 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.875231  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.876654  8891 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:09.870899  8747 rpcz_store.cc:276] 0709 13:01:06.852648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.869504 (+3016856us) service_pool.cc:224] Handling call
0709 13:01:09.870363 (+   859us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.872123  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:09.877379 11592 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:09.867626  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.857126 12196 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.878190  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.878509  8740 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:09.878968  8747 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:09.879446  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.880151  8740 raft_consensus.cc:2368] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:09.880518  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1901) took 1472 ms. Trace:
I20260709 13:01:09.869726  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.881001  8740 rpcz_store.cc:276] 0709 13:01:08.408612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.879211 (+1470599us) service_pool.cc:224] Handling call
0709 13:01:09.880470 (+  1259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:09.883436  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:09.880539  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:09.886756  8821 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:09.859179  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.868676 12330 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.887290  8891 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:09.877738  8818 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.843369 12439 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:09.889902  8895 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.915971 12330 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:09.917057 12428 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:09.917732  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:09.919935  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.846848 12444 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.946718  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.857853 12195 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.949477 12242 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.950210  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:09.961362  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 568) took 2448 ms (client timeout 2060 ms). Trace:
I20260709 13:01:09.887458 12408 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:09.879760  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.856959 12049 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:10.049816  9444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 661) took 4659 ms (client timeout 1625 ms). Trace:
W20260709 13:01:09.961835  8823 rpcz_store.cc:269] 0709 13:01:07.512928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.119327 (+606399us) service_pool.cc:224] Handling call
0709 13:01:09.961148 (+1841821us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b540004d8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:09.961274 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212316416}
I20260709 13:01:10.056638  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.058813  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 560) took 4622 ms. Trace:
I20260709 13:01:09.980242  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:09.983906  8898 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:01:10.059638  9444 rpcz_store.cc:269] 0709 13:01:05.385989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.398113 (+ 12124us) service_pool.cc:224] Handling call
0709 13:01:10.044991 (+4646878us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.045138 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6221527936}
I20260709 13:01:09.961764  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.061869  8894 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:10.062120  8891 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 6.
I20260709 13:01:10.058476  8895 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:09.961478  8821 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:10.063449  8745 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:09.972190  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1922) took 4028 ms. Trace:
I20260709 13:01:10.064177  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.064541  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:10.065181  8821 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:10.062599  8740 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:09.974696 12412 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 2.225s	user 0.013s	sys 0.009s
I20260709 13:01:09.987426  8819 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.064337  8889 rpcz_store.cc:276] 0709 13:01:05.944259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.944700 (+   441us) service_pool.cc:224] Handling call
0709 13:01:05.982765 (+ 38065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.253126 (+270361us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:07.944753 (+1691627us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.964402 (+ 19649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:07.964456 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:07.964501 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.971961 (+2007460us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:09.972008 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.972134 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1302644,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":102,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":1819,"spinlock_wait_cycles":621640192,"thread_start_us":1691263,"threads_started":2,"wal-append.queue_time_us":40577,"wal-append.run_cpu_time_us":1569,"wal-append.run_wall_time_us":1057909}
I20260709 13:01:09.854799 12406 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.065696  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.074750  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.074134  8818 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:10.079361  8748 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:10.078603  8815 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:10.005355 12466 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.083521 12466 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.088049  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.089320  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.089956  8740 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 6.
I20260709 13:01:10.090857  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.091802  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.092949  8740 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:10.060395  8893 rpcz_store.cc:276] 0709 13:01:05.436448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.437228 (+   780us) service_pool.cc:224] Handling call
0709 13:01:05.437565 (+   337us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:05.460146 (+ 22581us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:05.460395 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 13:01:05.728914 (+268519us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:06.512306 (+783392us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:06.514991 (+  2685us) log.cc:844] Serialized 78 byte log entry
0709 13:01:06.515131 (+   140us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:06.515169 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.515237 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.039923 (+2524686us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b1800302800. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1
0709 13:01:09.922125 (+882202us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b1800302800. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:09.933565 (+ 11440us) raft_consensus.cc:1685] finished
0709 13:01:09.933589 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.058749 (+125160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32242,"prepare.queue_time_us":1126,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":50400,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":12153,"spinlock_wait_cycles":68884096,"thread_start_us":265071,"threads_started":1,"wal-append.queue_time_us":303792}
I20260709 13:01:10.093236  8740 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:10.094595  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:09.948279 12439 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.094781  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.098544  8897 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:09.974179 12394 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.095180  8893 raft_consensus.cc:2376] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8. Re-sending same reply.
I20260709 13:01:10.094866  8740 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.111445  8740 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.110466  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.106103  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.112147  8896 raft_consensus.cc:2376] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:01:10.023471 12198 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:10.062353 12386 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.106904  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.111958  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.122926  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:10.123291  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1601) took 1032 ms. Trace:
I20260709 13:01:10.123692  8748 rpcz_store.cc:276] 0709 13:01:09.091362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.079991 (+988629us) service_pool.cc:224] Handling call
0709 13:01:10.123232 (+ 43241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.125141  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.125553  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.113186  8818 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:01:10.075294 12142 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.528s	user 0.006s	sys 0.014s
I20260709 13:01:10.134285 12300 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.981580 12242 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:10.134755  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1552) took 4552 ms (client timeout 1623 ms). Trace:
W20260709 13:01:10.135181  8820 rpcz_store.cc:269] 0709 13:01:05.582024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.582477 (+   453us) service_pool.cc:224] Handling call
0709 13:01:10.134577 (+4552100us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b5400075b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.134691 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5724898176}
I20260709 13:01:10.136596 12242 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:10.118808  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.138103  8821 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.138655  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 562) took 3511 ms. Trace:
I20260709 13:01:10.138948  8821 rpcz_store.cc:276] 0709 13:01:06.627751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.137643 (+3509892us) service_pool.cc:224] Handling call
0709 13:01:10.137932 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.138453 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.138606 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.140213  8821 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:10.140694  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1565) took 2046 ms. Trace:
I20260709 13:01:09.972093 12446 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.073555 12401 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.126681  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.140980  8821 rpcz_store.cc:276] 0709 13:01:08.093976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.139662 (+2045686us) service_pool.cc:224] Handling call
0709 13:01:10.139905 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.140498 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.140647 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.129324  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.128285  8892 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.133459 12443 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.148857  8818 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:10.145171  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:10.125967  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.147967  8748 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:10.131009 12207 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.029s	user 0.009s	sys 0.004s
I20260709 13:01:10.089010 12402 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:10.225055  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:10.201011  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.154997  8897 raft_consensus.cc:2376] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
W20260709 13:01:10.266775  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.316398  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:10.247550  9414 leader_election.cc:336] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.916s (SENT)
I20260709 13:01:10.321808  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:10.247519  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.322124  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.143092 12235 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.325279  8818 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:10.326036  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1566) took 2231 ms. Trace:
I20260709 13:01:10.225442  8747 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:10.327803  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80309.8us
I20260709 13:01:10.174379  8740 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:10.326359  8818 rpcz_store.cc:276] 0709 13:01:08.094289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.324799 (+2230510us) service_pool.cc:224] Handling call
0709 13:01:10.325115 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.325807 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.325987 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.328954  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 541) took 3480 ms. Trace:
I20260709 13:01:10.318651  8892 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:10.329749  8818 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.329525  8740 rpcz_store.cc:276] 0709 13:01:06.848326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.092571 (+3244245us) service_pool.cc:224] Handling call
0709 13:01:10.092842 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.328699 (+235857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.328898 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.330550  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1572) took 2234 ms. Trace:
I20260709 13:01:10.330967  8818 rpcz_store.cc:276] 0709 13:01:08.096104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.329337 (+2233233us) service_pool.cc:224] Handling call
0709 13:01:10.329593 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.330328 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.330500 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.331446  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.332326  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2204) took 2210 ms. Trace:
I20260709 13:01:10.332683  8818 rpcz_store.cc:276] 0709 13:01:08.121765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.331647 (+2209882us) service_pool.cc:224] Handling call
0709 13:01:10.331880 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.332035 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.332073 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.332113 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.332141 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.332192 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.332281 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.142831 12425 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.788s	user 0.000s	sys 0.011s
I20260709 13:01:10.334337  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2205) took 2212 ms. Trace:
I20260709 13:01:10.335067  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.334673  8818 rpcz_store.cc:276] 0709 13:01:08.122142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.333660 (+2211518us) service_pool.cc:224] Handling call
0709 13:01:10.333896 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.334058 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.334084 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.334123 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.334152 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.334200 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.334293 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.148463 12397 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.003s
I20260709 13:01:10.247197  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:10.297947 12185 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.000s
W20260709 13:01:10.223795 12396 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.000s
I20260709 13:01:10.340698  8891 raft_consensus.cc:2376] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10. Re-sending same reply.
I20260709 13:01:10.342247  8897 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:10.342413  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.336611 12398 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:10.341064 12436 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:01:10.344276  8818 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.344972  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.435623  8740 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:10.153892 12314 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.469544  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2206) took 2337 ms. Trace:
I20260709 13:01:10.224820 12446 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.298906 12460 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:10.436679  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:10.437297  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.470008  8818 rpcz_store.cc:276] 0709 13:01:08.132654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.338926 (+2206272us) service_pool.cc:224] Handling call
0709 13:01:10.344105 (+  5179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.469307 (+125202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.469492 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.340565 12425 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:10.474610  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.480794  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2207) took 2339 ms. Trace:
I20260709 13:01:10.483301  8818 rpcz_store.cc:276] 0709 13:01:08.141086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.479929 (+2338843us) service_pool.cc:224] Handling call
0709 13:01:10.480250 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.480425 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.480461 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.480511 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.480553 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.480621 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.480737 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.225766 12454 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.754s	user 0.000s	sys 0.014s
I20260709 13:01:10.485875  8740 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.500943  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1889) took 2096 ms. Trace:
I20260709 13:01:10.501499  8740 rpcz_store.cc:276] 0709 13:01:08.404808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.469624 (+2064816us) service_pool.cc:224] Handling call
0709 13:01:10.485692 (+ 16068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.500563 (+ 14871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.500867 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.134800 12300 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.434355 12237 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.325037 12448 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.502621  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.141742 12423 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.159507 12401 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.228587 12393 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.519271  8740 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:10.520165  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1891) took 2114 ms. Trace:
I20260709 13:01:10.520599  8740 rpcz_store.cc:276] 0709 13:01:08.405462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.502625 (+2097163us) service_pool.cc:224] Handling call
0709 13:01:10.518826 (+ 16201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.519804 (+   978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.520088 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.522629  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1892) took 2116 ms. Trace:
I20260709 13:01:10.523103  8740 rpcz_store.cc:276] 0709 13:01:08.405770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.521610 (+2115840us) service_pool.cc:224] Handling call
0709 13:01:10.522008 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.522220 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.522258 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.522318 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.522360 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.522430 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.522562 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.524884  8740 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:10.511375  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.477854 12402 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.341363 12397 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.419158 12185 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.527642  8891 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:10.530118 12353 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:10.534405 12235 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.515570 12237 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:10.538317 12470 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.410s	user 0.000s	sys 0.001s
I20260709 13:01:10.525727 12049 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.546861 12401 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.498996 12347 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.582557  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.559049  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.585199  8748 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:10.563897  8789 leader_election.cc:400] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:10.574184  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1895) took 2167 ms. Trace:
I20260709 13:01:10.585870  8740 rpcz_store.cc:276] 0709 13:01:08.406716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.524337 (+2117621us) service_pool.cc:224] Handling call
0709 13:01:10.524671 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.573814 (+ 49143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.574102 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.587239  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.588186  8740 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:10.589372  9415 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.602s (SENT)
I20260709 13:01:10.576511  8743 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:10.576710  8893 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:10.592108  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:10.594079  8821 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.602375  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 561) took 3975 ms. Trace:
I20260709 13:01:10.602841  8820 rpcz_store.cc:276] 0709 13:01:06.626520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.135989 (+3509469us) service_pool.cc:224] Handling call
0709 13:01:10.136304 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.136503 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:10.476260 (+339757us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.602006 (+125746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.602065 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:10.602116 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.602186 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.602320 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":464731,"threads_started":2}
I20260709 13:01:10.572592 12353 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:10.595453 12469 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:01:10.609066  8820 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:10.604364 12401 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:10.610097  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.610615  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1897) took 2203 ms. Trace:
I20260709 13:01:10.611852  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2222) took 1202 ms. Trace:
W20260709 13:01:10.612323  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 671) took 2208 ms (client timeout 2224 ms). Trace:
W20260709 13:01:10.618032 11390 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.112s	user 0.017s	sys 0.000s
I20260709 13:01:10.618462 12448 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:10.619311 12189 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.769s	user 0.006s	sys 0.002s
I20260709 13:01:10.620055 12185 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.620378  9448 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:10.620627 12423 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:10.620878 12195 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.002s
I20260709 13:01:10.632249 12401 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.602250 12235 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:01:10.609130 12393 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:10.622167  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1902) took 2213 ms. Trace:
I20260709 13:01:10.621837 12353 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.597401 12347 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.648563 12347 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:10.649117 12235 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:10.627663  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.627576  8889 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:10.631613  9442 rpcz_store.cc:269] 0709 13:01:08.403033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.403300 (+   267us) service_pool.cc:224] Handling call
0709 13:01:10.611108 (+2207808us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.611228 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654109056}
W20260709 13:01:10.650602  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.631440  8740 rpcz_store.cc:276] 0709 13:01:08.407363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.589027 (+2181664us) service_pool.cc:224] Handling call
0709 13:01:10.605832 (+ 16805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.610278 (+  4446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.610306 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.610346 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.610377 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.610437 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.610553 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":2354}
W20260709 13:01:10.651017  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.632458  8820 rpcz_store.cc:276] 0709 13:01:09.409653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.608540 (+1198887us) service_pool.cc:224] Handling call
0709 13:01:10.608898 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.611632 (+  2734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.611759 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.641487  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 535) took 4064 ms (client timeout 1853 ms). Trace:
W20260709 13:01:10.620990 12433 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.007s
I20260709 13:01:10.645213  8748 rpcz_store.cc:276] 0709 13:01:08.408913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.605141 (+2196228us) service_pool.cc:224] Handling call
0709 13:01:10.621660 (+ 16519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.621837 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.621860 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.621906 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.621937 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.621991 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.622112 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.650991  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 570) took 1916 ms. Trace:
I20260709 13:01:10.633178  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2183) took 1246 ms. Trace:
I20260709 13:01:10.653388  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.653568  8891 rpcz_store.cc:276] 0709 13:01:09.386513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.583922 (+1197409us) service_pool.cc:224] Handling call
0709 13:01:10.633099 (+ 49177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.653409  8889 rpcz_store.cc:276] 0709 13:01:08.734222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.626996 (+1892774us) service_pool.cc:224] Handling call
0709 13:01:10.627380 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.650637 (+ 23257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.650917 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.651849  8744 rpcz_store.cc:269] 0709 13:01:06.576753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.594822 (+ 18069us) service_pool.cc:224] Handling call
0709 13:01:08.693461 (+2098639us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:10.635856 (+1942395us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400462dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.641425 (+  5569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917695488}
I20260709 13:01:10.652684 12235 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:10.654940  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.663236  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:10.645967 12195 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.666064 12244 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.645581 12401 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.666239 12396 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:10.666518 12244 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:10.667021 12244 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.667824 12195 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.664304 12433 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.668972 12397 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.669482  8890 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.641336 12189 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.672362  8740 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:10.672978  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1595) took 1681 ms. Trace:
I20260709 13:01:10.673261  8740 rpcz_store.cc:276] 0709 13:01:08.991253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.651621 (+1660368us) service_pool.cc:224] Handling call
0709 13:01:10.672061 (+ 20440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.672695 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.672869 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.674700  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1598) took 1652 ms. Trace:
I20260709 13:01:10.675040  8740 rpcz_store.cc:276] 0709 13:01:09.021970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.673993 (+1652023us) service_pool.cc:224] Handling call
0709 13:01:10.674255 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.674419 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.674443 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.674483 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:10.674510 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.674561 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.674657 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.676352  8740 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:10.676786  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1599) took 1651 ms. Trace:
I20260709 13:01:10.677038  8740 rpcz_store.cc:276] 0709 13:01:09.025534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.675856 (+1650322us) service_pool.cc:224] Handling call
0709 13:01:10.676141 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.676612 (+   471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.676751 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.665526  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.678732  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.678947  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:10.679175  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1600) took 1601 ms. Trace:
I20260709 13:01:10.679347  8889 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:10.679493  8740 rpcz_store.cc:276] 0709 13:01:09.077663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.677916 (+1600253us) service_pool.cc:224] Handling call
0709 13:01:10.678707 (+   791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.678873 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.678907 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.678951 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:10.678983 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.679037 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.679132 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.681275  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1597) took 1673 ms. Trace:
I20260709 13:01:10.681658 12397 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:10.682216 12397 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.681895  8744 rpcz_store.cc:276] 0709 13:01:09.007471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.656320 (+1648849us) service_pool.cc:224] Handling call
0709 13:01:10.680675 (+ 24355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.680887 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.680923 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.680978 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.681024 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.681095 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.681210 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.682804 12397 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:10.684147  8744 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.684913  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1605) took 1370 ms. Trace:
I20260709 13:01:10.685631  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:10.689122  8744 rpcz_store.cc:276] 0709 13:01:09.314325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.683544 (+1369219us) service_pool.cc:224] Handling call
0709 13:01:10.683925 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.684633 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.684851 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.689465  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:10.638098 11390 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:10.680737  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1604) took 1438 ms. Trace:
I20260709 13:01:10.664435 12393 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:10.705865 12393 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:10.706284 12393 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.703774  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.671650 12353 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.709739  8889 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:10.710664  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.712204  8889 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:10.712785  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1926) took 1718 ms. Trace:
I20260709 13:01:10.714795  8889 rpcz_store.cc:276] 0709 13:01:08.994824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.711516 (+1716692us) service_pool.cc:224] Handling call
0709 13:01:10.711830 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.712570 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.712736 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.715770  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.716498  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1927) took 1721 ms. Trace:
I20260709 13:01:10.716854  8889 rpcz_store.cc:276] 0709 13:01:08.995160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.715592 (+1720432us) service_pool.cc:224] Handling call
0709 13:01:10.716029 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.716199 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.716222 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.716259 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:10.716293 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.716342 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.716451 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.718204  8889 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.718751  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1928) took 1723 ms. Trace:
I20260709 13:01:10.719453 12283 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.719822 12397 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.722282  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:10.722522  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.76543e+06us
I20260709 13:01:10.705318  8740 rpcz_store.cc:276] 0709 13:01:09.242732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.680190 (+1437458us) service_pool.cc:224] Handling call
0709 13:01:10.680427 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.680549 (+   122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.680564 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.680589 (+    25us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:10.680606 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.680639 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.680705 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.703616 12049 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:10.724684  8740 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:10.731958  8891 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:10.719141  8889 rpcz_store.cc:276] 0709 13:01:08.995476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.717756 (+1722280us) service_pool.cc:224] Handling call
0709 13:01:10.718041 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.718501 (+   460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.718692 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.704043  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.726308  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 750) took 1373 ms. Trace:
I20260709 13:01:10.724242  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.732594  9448 rpcz_store.cc:276] 0709 13:01:09.352865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.364212 (+ 11347us) service_pool.cc:224] Handling call
0709 13:01:10.611257 (+1247045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.639240 (+ 27983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.726050 (+ 86810us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540067cfd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.726231 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2867919744}
I20260709 13:01:10.716854  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.733989  8816 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.735642  8744 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:10.735924  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.736402  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1909) took 1336 ms. Trace:
W20260709 13:01:10.730566  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 158977us
I20260709 13:01:10.737046  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.678252 12233 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.738408  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:10.736233  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1606) took 1410 ms. Trace:
I20260709 13:01:10.738214  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:10.741351  8740 rpcz_store.cc:276] 0709 13:01:09.325452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.723701 (+1398249us) service_pool.cc:224] Handling call
0709 13:01:10.724041 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.732047 (+  8006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.736182 (+  4135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.737787  8889 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.743414  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1910) took 1343 ms. Trace:
I20260709 13:01:10.740984  8742 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:10.743515  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.702726 12185 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:10.744131  8740 rpcz_store.cc:276] 0709 13:01:09.399847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.742243 (+1342396us) service_pool.cc:224] Handling call
0709 13:01:10.742723 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.743083 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.743112 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.743156 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.743195 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.743254 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.743357 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.746230  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.736896  8744 rpcz_store.cc:276] 0709 13:01:09.399564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.735065 (+1335501us) service_pool.cc:224] Handling call
0709 13:01:10.735355 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.736121 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.736298 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.746750  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:10.745566  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1912) took 1319 ms. Trace:
I20260709 13:01:10.749189  8740 rpcz_store.cc:276] 0709 13:01:09.426005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.744923 (+1318918us) service_pool.cc:224] Handling call
0709 13:01:10.745125 (+   202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.745229 (+   104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.745243 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.745269 (+    26us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:10.745347 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.745409 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.745517 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:10.750126  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.746230  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.751690  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.752157  8820 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.734566 12347 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.758631 12436 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:10.737926 12422 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:01:10.746160 12185 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:10.756605 12330 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.760154 12185 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.757305 12433 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:10.756062 12189 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:10.758375 12397 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.758925 11390 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.759383 12347 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.762143 12347 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:10.762564  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1898) took 2354 ms (client timeout 2016 ms). Trace:
W20260709 13:01:10.762969  8749 rpcz_store.cc:269] 0709 13:01:08.407668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.183735 (+776067us) service_pool.cc:224] Handling call
0709 13:01:10.762361 (+1578626us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.762495 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2851980672}
I20260709 13:01:10.765146  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.765640 12443 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:10.766283  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.766608 12443 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:10.767045 12443 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:10.767685  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.767812 12443 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.768983 12443 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:10.769697  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2199) took 4419 ms (client timeout 1915 ms). Trace:
I20260709 13:01:10.788407  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:10.789011  9443 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.789824  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.769922 12347 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:10.770490  8819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2210) took 2589 ms (client timeout 1540 ms). Trace:
I20260709 13:01:10.792718  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:10.746030 12049 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:10.743153 12448 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:01:10.760169 12422 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:10.773260 12436 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:10.842700  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46809.5us
W20260709 13:01:10.843526  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:10.775157  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.844900  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.846386  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.847831  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.766834  8817 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.730227 12401 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:10.780237  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2208) took 2633 ms (client timeout 1849 ms). Trace:
W20260709 13:01:10.857605  8815 rpcz_store.cc:269] 0709 13:01:08.146596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.631231 (+484635us) service_pool.cc:224] Handling call
0709 13:01:10.780025 (+2148794us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.780163 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200380800}
W20260709 13:01:10.781661  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 539) took 4045 ms (client timeout 1924 ms). Trace:
I20260709 13:01:10.782998  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2181) took 1398 ms. Trace:
I20260709 13:01:10.768258  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:10.785378  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.861351  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:10.781155  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.790594  8889 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:10.863447  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.864691  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:10.786072  8822 rpcz_store.cc:269] 0709 13:01:06.350718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.353397 (+  2679us) service_pool.cc:224] Handling call
0709 13:01:10.769522 (+4416125us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b5400020dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.769625 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6760111488}
I20260709 13:01:10.865924  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:10.866345  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77348.4us
I20260709 13:01:10.866830  8822 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.867672  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:10.868003  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.869062  8822 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:10.771914 12466 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:10.784453  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1887) took 2380 ms. Trace:
I20260709 13:01:10.796753  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:10.872085  9449 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.872275  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.873818  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.875017  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.875967  8822 raft_consensus.cc:2410] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:10.876797  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.800484  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:10.877635  9448 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:10.879475  8745 rpcz_store.cc:276] 0709 13:01:08.404148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.334144 (+1929996us) service_pool.cc:224] Handling call
0709 13:01:10.387159 (+ 53015us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:10.433911 (+ 46752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.433965 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:10.484875 (+ 50910us) log.cc:844] Serialized 52 byte log entry
0709 13:01:10.609656 (+124781us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:10.783873 (+174217us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.784155 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.784229 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.784269 (+    40us) raft_consensus.cc:1685] finished
0709 13:01:10.784280 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.784401 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49820,"raft.queue_time_us":53988,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":91348,"thread_start_us":265937,"threads_started":2,"wal-append.queue_time_us":183973}
I20260709 13:01:10.882159  9443 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.882249  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.884002  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.773216  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.885542  8745 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:10.850329  8864 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.860078  8892 rpcz_store.cc:276] 0709 13:01:09.384912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.859204 (+474292us) service_pool.cc:224] Handling call
0709 13:01:10.782843 (+923639us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54003bd154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.782943 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602728704}
I20260709 13:01:10.886458  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:10.887207  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.753556 12331 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:10.792620  8819 rpcz_store.cc:269] 0709 13:01:08.180909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.281029 (+100120us) service_pool.cc:224] Handling call
0709 13:01:10.770329 (+2489300us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540041ffd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.770429 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3332559104}
I20260709 13:01:10.893031  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.894456  9449 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:10.849252  8820 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:10.895783  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.887027  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:10.896814  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:10.897500  8820 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:10.870565  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.819456  8816 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:10.885574  8744 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:10.776131  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:10.887756  8892 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:10.898342  8740 raft_consensus.cc:2376] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10. Re-sending same reply.
I20260709 13:01:10.898813  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:10.900125  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:10.859236  8746 rpcz_store.cc:269] 0709 13:01:06.736624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.736766 (+   142us) service_pool.cc:224] Handling call
0709 13:01:09.001397 (+2264631us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:10.781462 (+1780065us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b5400358254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.781592 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085884672}
I20260709 13:01:10.901522  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:10.901700  8822 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:10.902091  8746 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:10.902298  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2223) took 1492 ms. Trace:
I20260709 13:01:10.888026  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:10.886554  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:10.902654  8822 rpcz_store.cc:276] 0709 13:01:09.409889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.898053 (+1488164us) service_pool.cc:224] Handling call
0709 13:01:10.898394 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.902053 (+  3659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.902247 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.903108  9442 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:10.886332 12405 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:10.783936 12398 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:10.897640  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:10.915333  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:10.905083  8817 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:10.784308 12042 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:10.797163 12049 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:10.849673 12442 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:10.923215  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:10.924245  8819 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:10.911038  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:10.912019  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2227) took 1497 ms. Trace:
W20260709 13:01:10.909891  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.888298 12433 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:10.842892 12422 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:10.937508 12422 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:10.938241 12475 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:10.938017 12244 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.938835 12422 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.940193 12244 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.913184 12195 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:10.941165 12195 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:10.941576 12195 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:10.841485 12393 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:10.942917 12401 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.943711 12393 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:10.843086 12436 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:10.842785 12448 consensus_queue.cc:1243] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:10.915561  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:11.175060  8815 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:10.919360  8889 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:10.919790  9446 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:01:10.922212  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 575) took 1454 ms (client timeout 1924 ms). Trace:
I20260709 13:01:10.927152  8820 rpcz_store.cc:276] 0709 13:01:09.414657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.911250 (+1496593us) service_pool.cc:224] Handling call
0709 13:01:10.911531 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.911695 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.911727 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.911765 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:10.911795 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.911854 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.911965 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.929207  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:10.869582  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 81256.4us
I20260709 13:01:10.849696 12233 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 13:01:10.936362 12433 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.918286  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.177577  8820 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:10.945858  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:10.945616  8817 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:01:11.179812  8817 pending_rounds.cc:85] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 3
W20260709 13:01:11.176465  8824 rpcz_store.cc:269] 0709 13:01:09.467601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.538232 (+ 70631us) service_pool.cc:224] Handling call
0709 13:01:10.922008 (+1383776us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:10.922146 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586710400}
W20260709 13:01:10.989159  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.938851 12049 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:10.921332 12042 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:10.944416 12442 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:11.015568  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.027209  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 237510us
I20260709 13:01:11.189342  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:11.168035  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2189) took 1779 ms. Trace:
I20260709 13:01:10.856658 12397 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:01:10.917644 12425 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.168354  8823 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:11.168819  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.192269  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2502738809999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
W20260709 13:01:11.192772  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2404497659999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 13:01:10.886544 12330 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.195065  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2228) took 1780 ms. Trace:
I20260709 13:01:11.195978  8823 rpcz_store.cc:276] 0709 13:01:09.414862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.918369 (+1503507us) service_pool.cc:224] Handling call
0709 13:01:10.966317 (+ 47948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.194745 (+228428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.194993 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.170881  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.171358 12481 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:11.171308  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 256252us
I20260709 13:01:11.171681 12266 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.199429  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:10.910982 12466 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:11.172325  8893 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1940) took 2158 ms (client timeout 2791 ms). Trace:
W20260709 13:01:11.240774  8893 rpcz_store.cc:269] 0709 13:01:09.013629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.110138 (+1096509us) service_pool.cc:224] Handling call
0709 13:01:11.098064 (+987926us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b540046df54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.172262 (+ 74198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068390656}
I20260709 13:01:10.944020 12401 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.966796 12445 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.076s
W20260709 13:01:11.172510  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 274982us
W20260709 13:01:11.172878  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2180) took 1788 ms (client timeout 2246 ms). Trace:
W20260709 13:01:10.945462  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77889.8us
I20260709 13:01:11.252076  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:11.180008  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:11.254227  8891 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:01:11.252439  8894 rpcz_store.cc:269] 0709 13:01:09.384646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.869528 (+484882us) service_pool.cc:224] Handling call
0709 13:01:10.997882 (+1128354us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540045a6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.172820 (+174938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415510400}
I20260709 13:01:11.180438  8817 pending_rounds.cc:107] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:01:11.191227 12442 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.256151  8817 raft_consensus.cc:2889] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:11.187053 12433 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.257113  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1589) took 1635 ms. Trace:
I20260709 13:01:11.257658  8817 rpcz_store.cc:276] 0709 13:01:09.621705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.919467 (+1297762us) service_pool.cc:224] Handling call
0709 13:01:10.920053 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.256799 (+336746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.257043 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.259477  8817 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:11.179395  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1596) took 1326 ms. Trace:
I20260709 13:01:11.261509  8820 rpcz_store.cc:276] 0709 13:01:09.853310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.176857 (+1323547us) service_pool.cc:224] Handling call
0709 13:01:11.177231 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.179096 (+  1865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.179319 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.253384  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:11.080336 12283 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.190538  8897 rpcz_store.cc:276] 0709 13:01:09.388158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.334837 (+946679us) service_pool.cc:224] Handling call
0709 13:01:11.095988 (+761151us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b540019ebd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.167973 (+ 71985us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14790016}
I20260709 13:01:11.192407  8819 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:11.169610  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1567) took 3075 ms. Trace:
W20260709 13:01:11.193181  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1942406340000389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 13:01:11.270043  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1590) took 1648 ms. Trace:
I20260709 13:01:11.235553 12481 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.177469 12460 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:01:11.169983 12482 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.211839  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:11.273247  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61583.9us
W20260709 13:01:11.274115  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 555) took 5843 ms (client timeout 1790 ms). Trace:
I20260709 13:01:11.172022 12207 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:11.274514  8898 rpcz_store.cc:269] 0709 13:01:05.431234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.435207 (+  3973us) service_pool.cc:224] Handling call
0709 13:01:11.273929 (+5838722us) spinlock_profiling.cc:243] Waited 4.55 s on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.274052 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10461149952}
I20260709 13:01:11.239142  8824 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:10.844390 12189 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 13:01:11.242741  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64227.5us
I20260709 13:01:11.254565 12393 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:11.255661 12445 consensus_queue.cc:1243] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:11.177553 12399 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.179122 12353 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:11.244686 12466 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.195539 12330 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.911131 12331 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.191771 12425 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.255822  8894 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:11.330653  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1933) took 2319 ms. Trace:
I20260709 13:01:11.198233  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.331403  8894 rpcz_store.cc:276] 0709 13:01:09.011358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.254966 (+2243608us) service_pool.cc:224] Handling call
0709 13:01:11.255389 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.330286 (+ 74897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.330598 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.332050  8741 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:11.332635  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.269798  8821 rpcz_store.cc:276] 0709 13:01:08.094701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.325758 (+2231057us) service_pool.cc:224] Handling call
0709 13:01:10.326402 (+   644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.597068 (+270666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.098055 (+500987us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540001f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:11.169555 (+ 71500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587924224}
I20260709 13:01:11.270533  8819 rpcz_store.cc:276] 0709 13:01:09.621965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.965405 (+1343440us) service_pool.cc:224] Handling call
0709 13:01:11.192185 (+226780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.269779 (+ 77594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.269991 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.271359  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:11.251688 12401 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.313606  8898 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:01:11.313771  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 114209us
I20260709 13:01:11.314383 12492 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.315069 12490 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.270814  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:11.333101  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.369946  8741 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:11.282433 12399 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:11.286274 12466 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.276815 12207 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.278841 12486 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.370937  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.313009 12330 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.250327 12436 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.276144 12424 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:11.282124 12393 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:11.426403 12393 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.427865 12393 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.329715 12331 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.429786 12491 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.430277 12491 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:11.430634 12491 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:11.414283  9415 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.924s (SENT)
I20260709 13:01:11.377899  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1936) took 2365 ms. Trace:
I20260709 13:01:11.433594  8898 rpcz_store.cc:276] 0709 13:01:09.012330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.277066 (+2264736us) service_pool.cc:224] Handling call
0709 13:01:11.285068 (+  8002us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.377534 (+ 92466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.377824 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.436547 12049 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:11.337014  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65746.2us
W20260709 13:01:11.335409  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 575) took 1493 ms (client timeout 1756 ms). Trace:
I20260709 13:01:11.333876 12425 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:11.438091  8896 rpcz_store.cc:269] 0709 13:01:09.841783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.860678 (+ 18895us) service_pool.cc:224] Handling call
0709 13:01:11.335166 (+1474488us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54003bcc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.335313 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553384832}
I20260709 13:01:11.368611 12401 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.332690 12403 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.285326 12327 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:11.373366 12443 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.005s
I20260709 13:01:11.416163 12486 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.281160 12482 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:11.412011  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 160075us
W20260709 13:01:11.426450 12485 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:01:11.373103 12399 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:11.417471 12314 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:11.416242 12207 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.448841 12486 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.286526 12353 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.369120 12492 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:11.489514 12492 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.370339 12406 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.372102 12266 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.493817 12266 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.494257 12266 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:11.496985 12195 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.007s
I20260709 13:01:11.497457 12195 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.279649 12042 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.499496 12195 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.499620 12042 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.482945 12448 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:11.502321 12439 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.514083 12353 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.514578 12300 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:11.468255 12185 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.008s
I20260709 13:01:11.455440 12424 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:01:11.515538 12185 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.449273 12189 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:01:11.517671 12253 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.470245 12049 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.425239 12490 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.464093 12398 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:11.518533 12330 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.519376 12481 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.519779 12398 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.520670 12283 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:11.521144 12283 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.520712 12490 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.475221  8895 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:11.523398  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2162) took 2508 ms. Trace:
I20260709 13:01:11.523721  8895 rpcz_store.cc:276] 0709 13:01:09.014699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.474216 (+2459517us) service_pool.cc:224] Handling call
0709 13:01:11.474640 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.523141 (+ 48501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.523340 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.476691 12436 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.487123 12237 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:11.525424 12422 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.007s
W20260709 13:01:11.526052 12497 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:01:11.528133 12207 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.525041 12185 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.530223 12207 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.513579 12235 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.333493 12233 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:01:11.549222 12347 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.555483 12492 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.557482 12482 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.557228 12492 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.560678 12491 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:11.418965  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 138768us
I20260709 13:01:11.452997  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:11.453557  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2176) took 2070 ms (client timeout 1684 ms). Trace:
W20260709 13:01:11.532410  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 257577us
W20260709 13:01:11.719981  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.721236  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.609879  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:11.645984  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 527) took 5071 ms (client timeout 2082 ms). Trace:
I20260709 13:01:11.731678 12393 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:11.626061  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.618031  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.736578  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.620144  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1591) took 1770 ms. Trace:
W20260709 13:01:11.619925  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1888) took 3215 ms (client timeout 2137 ms). Trace:
I20260709 13:01:11.650722  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.644749  8889 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:11.616065  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1903) took 2218 ms (client timeout 2193 ms). Trace:
I20260709 13:01:11.739387  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2170) took 2479 ms. Trace:
I20260709 13:01:11.739928  8889 rpcz_store.cc:276] 0709 13:01:09.260224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.643526 (+2383302us) service_pool.cc:224] Handling call
0709 13:01:11.644159 (+   633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.738896 (+ 94737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.739258 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:11.739820  8747 rpcz_store.cc:269] 0709 13:01:09.397746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.879539 (+481793us) service_pool.cc:224] Handling call
0709 13:01:11.615449 (+1735910us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540013bfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.615798 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381992832}
I20260709 13:01:11.742434  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.743938  8746 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:11.750528  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:11.744174  8747 raft_consensus.cc:2410] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:11.644898  8818 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:11.373768  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 103035us
I20260709 13:01:11.598593  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1596) took 2606 ms. Trace:
W20260709 13:01:11.731186  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 121648us
I20260709 13:01:11.646405  8895 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:11.647702  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:11.593029  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.731163  8743 rpcz_store.cc:269] 0709 13:01:06.574304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.574415 (+   111us) service_pool.cc:224] Handling call
0709 13:01:11.645382 (+5070967us) spinlock_profiling.cc:243] Waited 3.99 s on lock 0x7b5400465ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.645639 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9170768768}
I20260709 13:01:11.646236  8896 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:11.611012  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.665964 12235 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.761525  8745 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:11.764640  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2221) took 2352 ms. Trace:
I20260709 13:01:11.765753  8744 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:11.765961  8818 rpcz_store.cc:276] 0709 13:01:09.409400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.508552 (+1099152us) service_pool.cc:224] Handling call
0709 13:01:11.643867 (+1135315us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:11.644383 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.760283 (+115900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.761183 (+   900us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74131328}
I20260709 13:01:11.767541  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:11.620417  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 263195us
W20260709 13:01:11.738132  8742 rpcz_store.cc:269] 0709 13:01:08.404523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.473128 (+1068605us) service_pool.cc:224] Handling call
0709 13:01:11.619369 (+2146241us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400160dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.619658 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2913557376}
I20260709 13:01:11.592798  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.774120 12486 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.778980  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:11.754074  8748 rpcz_store.cc:276] 0709 13:01:08.991518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.653424 (+1661906us) service_pool.cc:224] Handling call
0709 13:01:10.653741 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.664629 (+ 10888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.664660 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.679447 (+ 14787us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.077074 (+397627us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:11.077137 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.077244 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.597886 (+520642us) raft_consensus.cc:1685] finished
0709 13:01:11.597925 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.598309 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3957,"thread_start_us":339417,"threads_started":1,"wal-append.queue_time_us":580653}
W20260709 13:01:11.782919  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 339116us
I20260709 13:01:11.783561  8741 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:11.784293  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.785737 12492 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:11.786937  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 333963us
I20260709 13:01:11.788216  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:11.784857  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2168) took 2768 ms. Trace:
I20260709 13:01:11.791062 12452 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:11.784287  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1917) took 1013 ms. Trace:
I20260709 13:01:11.777226 12491 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.737795  8815 rpcz_store.cc:276] 0709 13:01:09.849563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.175761 (+1326198us) service_pool.cc:224] Handling call
0709 13:01:11.194836 (+ 19075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.271042 (+ 76206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.271073 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.271508 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.619045 (+347537us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:11.619190 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.619378 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.619468 (+    90us) raft_consensus.cc:1685] finished
0709 13:01:11.619487 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.619826 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75832,"raft.queue_time_us":75957,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":12591,"thread_start_us":286046,"threads_started":1,"wal-append.queue_time_us":173194}
I20260709 13:01:11.789793  8895 rpcz_store.cc:276] 0709 13:01:09.016611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.524452 (+2507841us) service_pool.cc:224] Handling call
0709 13:01:11.524760 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.778477 (+253717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.784521 (+  6044us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400249854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:11.784776 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278322048}
I20260709 13:01:11.792140  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:11.792841  8741 rpcz_store.cc:276] 0709 13:01:10.770659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.422039 (+651380us) service_pool.cc:224] Handling call
0709 13:01:11.784162 (+362123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.800973  8746 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.798915  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1920) took 1021 ms. Trace:
I20260709 13:01:11.799765  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.804504  8749 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9.
I20260709 13:01:11.802465  8748 rpcz_store.cc:276] 0709 13:01:10.771399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.783904 (+1012505us) service_pool.cc:224] Handling call
0709 13:01:11.792970 (+  9066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.806313  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1638) took 1025 ms. Trace:
I20260709 13:01:11.811942  8644 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 3 to 7, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 7 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:11.806679  8749 rpcz_store.cc:276] 0709 13:01:10.780727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.646990 (+866263us) service_pool.cc:224] Handling call
0709 13:01:11.806260 (+159270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.818394  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1639) took 1025 ms. Trace:
I20260709 13:01:11.819715  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1635) took 1034 ms. Trace:
I20260709 13:01:11.820210  8743 rpcz_store.cc:276] 0709 13:01:10.779999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.771784 (+991785us) service_pool.cc:224] Handling call
0709 13:01:11.814436 (+ 42652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.818883  8744 rpcz_store.cc:276] 0709 13:01:10.780962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.769115 (+988153us) service_pool.cc:224] Handling call
0709 13:01:11.806100 (+ 36985us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.821169  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:11.830667 12189 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:11.831413 12189 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:11.831487  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:11.832257  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:11.828430  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:11.829156  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.834936  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1935) took 2822 ms. Trace:
I20260709 13:01:11.827224  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:11.833241 12436 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.850924  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1631) took 1071 ms. Trace:
I20260709 13:01:11.852384  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1933) took 1062 ms. Trace:
I20260709 13:01:11.852896  8747 rpcz_store.cc:276] 0709 13:01:10.789997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.771726 (+981729us) service_pool.cc:224] Handling call
0709 13:01:11.852309 (+ 80583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.851473  8745 rpcz_store.cc:276] 0709 13:01:10.779008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.771878 (+992870us) service_pool.cc:224] Handling call
0709 13:01:11.850839 (+ 78961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.802695 12452 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:11.836202 12237 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.849472 12436 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:11.825712 12266 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.834730 12207 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.845924 12486 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:11.859670  8889 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:11.833715 12442 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.563s	user 0.016s	sys 0.001s
I20260709 13:01:11.830816 12198 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.860687 12207 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:11.873601 12198 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.874117 12207 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.874332 12198 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.833678  8890 rpcz_store.cc:269] 0709 13:01:09.383548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.831326 (+447778us) service_pool.cc:224] Handling call
0709 13:01:11.453412 (+1622086us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b540046ee54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:11.453498 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1925772544}
I20260709 13:01:11.845180  8897 rpcz_store.cc:276] 0709 13:01:09.012054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.269780 (+2257726us) service_pool.cc:224] Handling call
0709 13:01:11.270094 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.270288 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:11.655134 (+384846us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.833883 (+178749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:11.833999 (+   116us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:11.834117 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.834246 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.834628 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69191,"prepare.queue_time_us":259480,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":69386,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"thread_start_us":486297,"threads_started":2,"wal-append.queue_time_us":174221}
I20260709 13:01:11.878903  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1942) took 2864 ms. Trace:
I20260709 13:01:11.879375  8896 rpcz_store.cc:276] 0709 13:01:09.014250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.444348 (+2430098us) service_pool.cc:224] Handling call
0709 13:01:11.444748 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.779428 (+334680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.878629 (+ 99201us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:11.878828 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461759744}
I20260709 13:01:11.859032 12436 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.881445  8822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.008s
W20260709 13:01:11.881938  8822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.009s
I20260709 13:01:11.882726  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:11.883402  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 51359.7us
I20260709 13:01:11.885121 12330 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.885444  8863 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:11.886188 12424 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:11.837802 12445 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:11.886896 12424 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.887548 12445 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:11.878980  8897 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:11.888991 12445 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.890060  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2177) took 2506 ms. Trace:
I20260709 13:01:11.833796 12439 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:01:11.890465 12445 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:11.890743 12509 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:01:11.891443 12494 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:11.891021  8644 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:11.891916 12494 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:11.892480  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.893916 12406 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.894601 12406 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.896291  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:11.896942  8824 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
W20260709 13:01:11.897979  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 93538.9us
I20260709 13:01:11.899111  8821 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:11.869877  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 124551us
I20260709 13:01:11.899681  8824 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:11.889505 12492 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.897910  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:11.881006  8896 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:11.867703 12237 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.900689  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2171) took 2545 ms. Trace:
I20260709 13:01:11.890455  8897 rpcz_store.cc:276] 0709 13:01:09.383819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.878186 (+2494367us) service_pool.cc:224] Handling call
0709 13:01:11.878599 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.889727 (+ 11128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.889975 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.904484  8889 rpcz_store.cc:276] 0709 13:01:09.354953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.761022 (+2406069us) service_pool.cc:224] Handling call
0709 13:01:11.798571 (+ 37549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.866848 (+ 68277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.900640 (+ 33792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.869145  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:11.897804  8820 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:11.908974  8742 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:11.893014  8644 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:11.910632  8818 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:11.912793  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2178) took 2524 ms. Trace:
I20260709 13:01:11.913259  8896 rpcz_store.cc:276] 0709 13:01:09.384132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.880371 (+2496239us) service_pool.cc:224] Handling call
0709 13:01:11.880786 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.906724 (+ 25938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.908682 (+  1958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.914337  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1634) took 1134 ms. Trace:
I20260709 13:01:11.914664  8742 rpcz_store.cc:276] 0709 13:01:10.779744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.795223 (+1015479us) service_pool.cc:224] Handling call
0709 13:01:11.914292 (+119069us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:11.918179  9414 leader_election.cc:336] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.670s (SENT)
W20260709 13:01:11.919790  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.924s (SENT)
I20260709 13:01:11.919920  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:11.920460  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:11.922479 12492 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.923959 12195 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.924778 12195 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:11.925565  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65810.7us
I20260709 13:01:11.927752 12195 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:11.928134  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:11.928403 12195 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:11.929072 12237 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.833186 12189 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.932524  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:11.932694 12189 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:11.934872 12266 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:11.918090  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49161.5us
I20260709 13:01:11.939160 12425 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.940362  8889 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:11.940433  8740 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:11.939200  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:11.981882  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53898.8us
W20260709 13:01:11.941226 12331 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.511s	user 0.003s	sys 0.011s
W20260709 13:01:11.942034  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 181760us
I20260709 13:01:11.944968  8820 pending_rounds.cc:85] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 3
I20260709 13:01:11.945362  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:11.945488 12492 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.945578  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:11.893152 12494 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:11.946146  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:11.987072  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1947) took 2106 ms. Trace:
I20260709 13:01:11.990051  8889 rpcz_store.cc:276] 0709 13:01:09.872458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.939387 (+2066929us) service_pool.cc:224] Handling call
0709 13:01:11.939886 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.968767 (+ 28881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.979011 (+ 10244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.905948 12442 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:11.950155  8897 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:11.950229  8896 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:01:11.950623  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:11.953545  8748 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:11.945658  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:11.955314  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:11.956250 12433 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.012s
I20260709 13:01:11.993134  8748 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 4
I20260709 13:01:12.002642  8748 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:11.993764  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1946) took 2121 ms. Trace:
I20260709 13:01:12.003721  8896 rpcz_store.cc:276] 0709 13:01:09.872201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.949369 (+2077168us) service_pool.cc:224] Handling call
0709 13:01:11.949977 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.993453 (+ 43476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.993706 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62592}
I20260709 13:01:11.994181  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 573) took 2153 ms. Trace:
I20260709 13:01:11.958967 12491 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:11.959147  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.961925  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2225) took 2547 ms. Trace:
I20260709 13:01:11.963752  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1929) took 2959 ms. Trace:
I20260709 13:01:11.965248 12424 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:11.950066 12406 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.020289  8823 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:11.918143  8644 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 1 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 6 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:11.987154  8820 pending_rounds.cc:107] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:12.003019  8748 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:11.996920  8746 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:12.006115  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.006475 12300 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:12.007928  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1927) took 1234 ms. Trace:
I20260709 13:01:12.009459 12399 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.009918 12494 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.992107 12237 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.011593 12436 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.013366  8816 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:12.016383  8824 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:12.005213  8897 rpcz_store.cc:276] 0709 13:01:09.841220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.949369 (+2108149us) service_pool.cc:224] Handling call
0709 13:01:11.949855 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.993945 (+ 44090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.994131 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.952991 12393 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:12.018222  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 220533us
I20260709 13:01:12.021157  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1930) took 1246 ms. Trace:
I20260709 13:01:12.016803 12207 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.020841  8820 raft_consensus.cc:2889] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:12.021817  8815 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.056454  8746 rpcz_store.cc:276] 0709 13:01:10.775124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.750018 (+974894us) service_pool.cc:224] Handling call
0709 13:01:12.021102 (+271084us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.058348  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.022574  8740 rpcz_store.cc:276] 0709 13:01:10.773696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.187574 (+413878us) service_pool.cc:224] Handling call
0709 13:01:12.007687 (+820113us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54001c8254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:12.007837 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900732288}
I20260709 13:01:12.017469 12314 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:12.056499  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61638us
I20260709 13:01:12.018362  8893 rpcz_store.cc:276] 0709 13:01:09.004312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.242236 (+2237924us) service_pool.cc:224] Handling call
0709 13:01:11.242556 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.451203 (+208647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.451245 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.460030 (+  8785us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.845109 (+385079us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:11.859902 (+ 14793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.900264 (+ 40362us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.960965 (+ 60701us) raft_consensus.cc:1685] finished
0709 13:01:11.960986 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.963557 (+  2571us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71095,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"thread_start_us":318459,"threads_started":2,"wal-append.queue_time_us":173618}
I20260709 13:01:12.062685  8821 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:12.018251  8822 rpcz_store.cc:276] 0709 13:01:09.414203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.903455 (+1489252us) service_pool.cc:224] Handling call
0709 13:01:10.903739 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.175724 (+271985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.175775 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.188131 (+ 12356us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.882413 (+694282us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:11.882500 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.882602 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.961697 (+ 79095us) raft_consensus.cc:1685] finished
0709 13:01:11.961718 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.961867 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14164,"raft.queue_time_us":52650,"raft.run_cpu_time_us":11373,"raft.run_wall_time_us":204712,"thread_start_us":766714,"threads_started":2,"wal-append.queue_time_us":596644}
W20260709 13:01:12.064504  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.064752  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.065475  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:12.065559  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.066584  8896 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:12.066358  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64687.9us
I20260709 13:01:12.022300 12300 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:12.072976  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.066901 12237 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.054596 12195 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:12.075043  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119964us
I20260709 13:01:12.080004  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.075593  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:12.070712  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:12.080775  8896 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
W20260709 13:01:12.081535 12283 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.540s	user 0.000s	sys 0.019s
I20260709 13:01:12.083531  8898 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:12.083464 12283 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.084818  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.085125  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1941) took 3071 ms. Trace:
I20260709 13:01:11.996104 12331 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.061810 12314 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.087169  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:12.085614  8898 rpcz_store.cc:276] 0709 13:01:09.013877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.434736 (+2420859us) service_pool.cc:224] Handling call
0709 13:01:11.435097 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.084130 (+649033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.084839 (+   709us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b5400247a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:12.085068 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466393344}
I20260709 13:01:12.093065 12266 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:12.094519 12331 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.095186 12235 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:12.100180 12235 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:12.095769  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 334288us
I20260709 13:01:12.096916  9444 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:12.098371 12398 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:12.078392 12300 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:12.102265  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.101631 12198 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.116420 12198 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:12.118265  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.109706  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.110234  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.108812  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.022419 12433 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.127596  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:12.137091  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.137614  8644 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.138821  8892 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:12.139531 12436 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:12.140132  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1962) took 1349 ms. Trace:
I20260709 13:01:12.140654  8823 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:12.141124  8894 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:12.141983  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:12.146958  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 727) took 1334 ms. Trace:
I20260709 13:01:12.140462  8644 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 6, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:12.146656  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.153671  8817 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:12.154803  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:12.155290 12401 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.707s	user 0.003s	sys 0.014s
I20260709 13:01:12.157116 12049 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:12.157637  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.158430  8893 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.147329  9444 rpcz_store.cc:276] 0709 13:01:10.787241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.787352 (+   111us) service_pool.cc:224] Handling call
0709 13:01:10.787647 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.120207 (+1332560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.120994 (+   787us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:12.121255 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010532224}
W20260709 13:01:12.161717  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77049.7us
I20260709 13:01:12.140311 12436 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:12.162424 12433 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.162693  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.162438 12436 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.163592 12436 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.163671 12393 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.140589  8892 rpcz_store.cc:276] 0709 13:01:10.790372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.928923 (+138551us) service_pool.cc:224] Handling call
0709 13:01:12.139862 (+1210939us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b540015e354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:12.140032 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209735936}
I20260709 13:01:12.165334  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:12.166224 12207 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.166391  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:12.167133  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:12.167651  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.168349 12424 raft_consensus.cc:2764] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:12.165410  8896 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.169904  8815 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:12.169605  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:12.168331  8892 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.171558  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:12.102778 12314 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.155956 12401 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:12.193696  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.195551  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1629) took 1417 ms. Trace:
I20260709 13:01:12.196094  8748 rpcz_store.cc:276] 0709 13:01:10.778415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.820747 (+1042332us) service_pool.cc:224] Handling call
0709 13:01:11.910517 (+ 89770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.098444 (+187927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.188278 (+ 89834us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b54000b5c50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:12.195484 (+  7206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119726464}
W20260709 13:01:12.202521  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.202760  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:12.203266  8816 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:12.125669  8891 raft_consensus.cc:2955] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 4 to 7, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:12.170845 12399 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.105501 12235 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.145680 12398 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:12.217733  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:12.218289  8898 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:12.218400  8824 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:12.187642 12198 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.169432 12425 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:12.221264  8889 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:12.171865 12283 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:12.227108  8821 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:12.209461 12452 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.234825  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:12.234956  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61768.9us
I20260709 13:01:12.221211 12393 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:12.232590  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:12.234598  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:12.243696  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 153500us
W20260709 13:01:12.257481  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.243183  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:12.249686  9414 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.090s (SENT)
W20260709 13:01:12.261476  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66212.8us
I20260709 13:01:12.266409  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.267143  8817 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:12.260771  8898 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:12.266665  8893 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:12.259778 12233 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:01:12.262105 12393 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.272696 12393 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:12.257710 12466 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.833s	user 0.004s	sys 0.012s
I20260709 13:01:12.274405 12466 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.274925 12189 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:12.276569 12466 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.276837  8897 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:12.277499  8895 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:12.240100 12452 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.235026 12198 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.278105 12452 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.278206 12198 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.278594  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:12.280056  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:12.232949 12266 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:01:12.280921 12491 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:12.249709  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 310761us
I20260709 13:01:12.281872 12237 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.282158  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:12.282763  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:12.245316  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1931) took 1469 ms. Trace:
I20260709 13:01:12.283629  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:12.259507  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:12.281800  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:12.287443  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.254664  8889 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:12.261641  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:12.268252  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1955) took 1639 ms. Trace:
I20260709 13:01:12.296546  8898 rpcz_store.cc:276] 0709 13:01:10.628832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.259967 (+1631135us) service_pool.cc:224] Handling call
0709 13:01:12.260420 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.267963 (+  7543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.268184 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:12.298303  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49089.3us
I20260709 13:01:12.250172 12398 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.299177  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:12.299515  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.298331 12425 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:12.300976 12353 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:12.277130 12189 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:01:12.302278  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1948) took 2429 ms. Trace:
I20260709 13:01:12.302739  8889 rpcz_store.cc:276] 0709 13:01:09.872756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.253781 (+2381025us) service_pool.cc:224] Handling call
0709 13:01:12.254282 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.294290 (+ 40008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.302224 (+  7934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.303859  8897 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:12.283527  8741 rpcz_store.cc:276] 0709 13:01:10.775401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.815179 (+1039778us) service_pool.cc:224] Handling call
0709 13:01:11.887124 (+ 71945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.887829 (+   705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.887869 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.888742 (+   873us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.993840 (+105098us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:11.993893 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.993983 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.245070 (+251087us) raft_consensus.cc:1685] finished
0709 13:01:12.245088 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.245223 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":225,"thread_start_us":104604,"threads_started":1,"wal-append.queue_time_us":199370}
I20260709 13:01:12.242861 12283 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:12.313086  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53713.3us
I20260709 13:01:12.285106 12406 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:12.281818 12491 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:12.300602 12482 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.303537  8892 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:12.309022 12195 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.330312  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.301837 12399 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:01:12.351763  8896 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:12.324111 12482 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.301513 12331 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:12.330854 12195 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.348033 12049 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:12.369058  8893 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:12.369880  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1961) took 1626 ms. Trace:
I20260709 13:01:12.370204  8893 rpcz_store.cc:276] 0709 13:01:10.743505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.303753 (+1560248us) service_pool.cc:224] Handling call
0709 13:01:12.352493 (+ 48740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.369587 (+ 17094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.369820 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.352906 12042 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:12.358772 12425 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:12.359016 12406 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:12.368834 12237 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.371289 12195 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.359378 12482 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:12.394755  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112756us
I20260709 13:01:12.397722  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2172) took 3035 ms. Trace:
I20260709 13:01:12.398116  8895 rpcz_store.cc:276] 0709 13:01:09.362219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.847839 (+2485620us) service_pool.cc:224] Handling call
0709 13:01:11.879356 (+ 31517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.277910 (+398554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.397487 (+119577us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54000843d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:12.397662 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915364224}
W20260709 13:01:12.399618  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89401us
I20260709 13:01:12.402036  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2164) took 3386 ms. Trace:
I20260709 13:01:12.405370 12185 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.405831 12442 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.406796 12522 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:12.407636 12531 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:12.408094 12531 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:12.408476 12531 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:12.409405 12526 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:12.363314 12283 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.402534  8894 rpcz_store.cc:276] 0709 13:01:09.015247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.487333 (+2472086us) service_pool.cc:224] Handling call
0709 13:01:12.140851 (+653518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.146616 (+  5765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.394651 (+248035us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b540015e350. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:12.401960 (+  7309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502100736}
W20260709 13:01:12.419917  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 136426us
I20260709 13:01:12.431085 12049 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:12.431635 12049 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.433483  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:12.435940 12283 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.438339 12533 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:01:12.441982 12523 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:01:12.442611  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 129023us
I20260709 13:01:12.442787  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:12.442962  9446 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:12.443608  9449 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:12.444406  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:12.445379  9442 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:01:12.405872 12185 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.441721  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:12.447876  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.443485  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:12.450371  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63454.2us
I20260709 13:01:12.414809 12442 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:12.414356 12522 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:12.445906 12482 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.372304  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31c225a50048451db8964738c5db7540"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:01:12.441655  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:12.471292  8898 raft_consensus.cc:606] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:12.471714  8898 raft_consensus.cc:614] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:12.452071 12185 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:12.477536  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.412227 12436 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.482134  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.534037  9444 raft_consensus.cc:2368] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10.
I20260709 13:01:12.485787  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.472740  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:12.490096  8889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "011b370821004185a003be210a525c01"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:12.500996  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45066 (request call id 0) took 3069 ms. Trace:
I20260709 13:01:12.445379 12524 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:12.535019  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2175) took 3151 ms. Trace:
I20260709 13:01:12.536149  8898 rpcz_store.cc:276] 0709 13:01:09.420576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.321785 (+2901209us) service_pool.cc:224] Handling call
0709 13:01:12.472128 (+150343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:12.539999  8816 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:12.536748  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:12.532996  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 232865us
I20260709 13:01:12.537930  8889 raft_consensus.cc:606] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:12.539119  9445 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.454900 12042 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:12.449153 12300 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.601883  9442 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.484903 12442 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:12.602733  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 63609.7us
W20260709 13:01:12.605520  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.489456 12423 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:12.604763  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54350us
W20260709 13:01:12.617856  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.618881  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.621915  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.627655  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.619319  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 185677us
I20260709 13:01:12.491410 12233 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:12.540305  8890 rpcz_store.cc:276] 0709 13:01:09.383213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.877734 (+2494521us) service_pool.cc:224] Handling call
0709 13:01:11.878261 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.878867 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.878901 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.879470 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.996981 (+117511us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:12.050197 (+ 53216us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.050498 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.050596 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.534803 (+484207us) raft_consensus.cc:1685] finished
0709 13:01:12.534823 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.534956 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52961,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":1481,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":7312,"thread_start_us":117052,"threads_started":1,"wal-append.queue_time_us":560097}
I20260709 13:01:12.492136 12452 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.676091  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:12.645694  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:12.684686  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 200444us
I20260709 13:01:12.635454 12423 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.647651 12233 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:12.487475 12406 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.688745  9449 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:12.498508 12425 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.609937 12399 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:12.693082  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 325809us
W20260709 13:01:12.698215  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.498054 12522 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.686736 12423 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.687222 12491 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:12.704283  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:12.716157  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 295983us
I20260709 13:01:12.691382 12353 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:12.710433  9443 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:12.719743  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:12.739532  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 93335.8us
I20260709 13:01:12.751240  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:12.756932  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 507394us
I20260709 13:01:12.759506  8816 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:12.764392  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.764338  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:12.771189 12542 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:12.771960  8815 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:12.776588  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2169) took 3744 ms. Trace:
I20260709 13:01:12.771699  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.731076 12401 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.718323 12198 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.779040  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:12.820904  8647 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 300) took 26695 ms (client timeout 15000 ms). Trace:
I20260709 13:01:12.772639  9449 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:12.966504  9442 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:01:12.969739  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.911257  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 147597us
I20260709 13:01:12.957124  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:12.989714  8815 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:12.949985  9445 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:12.953380  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 205267us
I20260709 13:01:12.957299  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:13.016577  8823 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:12.958602  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 254748us
W20260709 13:01:12.780082 12525 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:01:13.017545  9444 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:12.798625 12283 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:12.799036 12442 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:12.950081  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:13.018843  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.014272  8815 raft_consensus.cc:606] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:12.921154 12314 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.688s	user 0.004s	sys 0.008s
I20260709 13:01:12.777216  8891 rpcz_store.cc:276] 0709 13:01:09.016877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.642204 (+2625327us) service_pool.cc:224] Handling call
0709 13:01:11.643525 (+  1321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.644727 (+  1202us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:11.656132 (+ 11405us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.866803 (+210671us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:12.052047 (+185244us) log.cc:844] Serialized 564 byte log entry
0709 13:01:12.052255 (+   208us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:12.752921 (+700666us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.760019 (+  7098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.760234 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.760352 (+   118us) raft_consensus.cc:1685] finished
0709 13:01:12.760371 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.761061 (+   690us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6456,"prepare.queue_time_us":19208,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":141,"raft.queue_time_us":14236,"raft.run_cpu_time_us":801,"raft.run_wall_time_us":59056,"thread_start_us":205164,"threads_started":1,"wal-append.queue_time_us":414390}
W20260709 13:01:12.780818 12534 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:01:12.765560 12233 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.038132  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:13.039047 12548 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:13.045725  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:13.075388  8817 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:01:13.076797  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.079298 12532 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.079634 12423 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.033077 12539 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:13.080621 12185 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:12.821141 12398 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:12.772325 12439 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.920414 12331 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.756587 12538 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.062433 12540 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.063120  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:12.810527 12436 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:12.806285 12452 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:12.951615 12300 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:13.026947  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.784660 12433 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.064221  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:13.064004 12237 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.779376 12491 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.073221  9448 raft_consensus.cc:2368] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10.
I20260709 13:01:13.073465  9440 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:13.080400 12542 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:12.798808 12195 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:13.049718  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 58073.5us
I20260709 13:01:12.950420 12543 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 13:01:13.081346 12548 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:12.953336  8647 rpcz_store.cc:269] 0709 13:00:46.122094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.132464 (+ 10370us) service_pool.cc:224] Handling call
0709 13:01:12.817199 (+26684735us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:03.563874 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:03.564564 (+   690us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:03.564615 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:01:03.564634 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:03.566185 (+  1551us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:03.566261 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:03.566290 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:01:03.650847 (+ 84557us) tablet.cc:701] Row locks acquired
0709 13:01:03.650893 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:01:03.651070 (+   177us) write_op.cc:270] Start()
0709 13:01:03.651217 (+   147us) write_op.cc:276] Timestamp: P: 1783602063650996 usec, L: 0
0709 13:01:03.651235 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:01:03.651981 (+   746us) log.cc:844] Serialized 6764 byte log entry
0709 13:01:03.995366 (+343385us) op_driver.cc:464] REPLICATION: finished
0709 13:01:04.261671 (+266305us) write_op.cc:301] APPLY: starting
0709 13:01:04.274170 (+ 12499us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:04.360593 (+ 86423us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:04.378943 (+ 18350us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:04.498670 (+119727us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:04.499173 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:01:04.499228 (+    55us) write_op.cc:312] APPLY: finished
0709 13:01:04.499731 (+   503us) log.cc:844] Serialized 119 byte log entry
0709 13:01:04.537439 (+ 37708us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:04.552823 (+ 15384us) write_op.cc:454] Released schema lock
0709 13:01:04.574444 (+ 21621us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:04.590146 (+ 15702us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:04.590188 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":423496,"reported_tablets":47,"spinlock_wait_cycles":1359872,"child_traces":[["op",{"apply.queue_time_us":230731,"apply.run_cpu_time_us":6653,"apply.run_wall_time_us":347671,"num_ops":17,"prepare.queue_time_us":135361,"prepare.run_cpu_time_us":16249,"prepare.run_wall_time_us":257195,"raft.queue_time_us":117882,"raft.run_cpu_time_us":15586,"raft.run_wall_time_us":168281,"replication_time_us":342277,"thread_start_us":355413,"threads_started":4,"wal-append.queue_time_us":199678,"wal-append.run_cpu_time_us":4938,"wal-append.run_wall_time_us":1649624}]]}
I20260709 13:01:12.788445 12533 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:13.110531  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.144558  9445 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:12.815251 12482 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:12.744890 12536 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.529s
I20260709 13:01:12.803249 12531 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.079743 12532 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.127315  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:13.125474  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:12.766016 12393 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.115736 12398 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:13.119777 12331 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.162572  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:12.757326 12042 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:13.138800  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60209.1us
I20260709 13:01:12.782689 12353 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.070359 12283 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:13.154634  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.970250 12524 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:13.142529 12436 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:13.137064  9449 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:13.156181 12049 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.000s
W20260709 13:01:13.169210  8695 auto_rebalancer.cc:893] Removing replace marker failed: Replica 7fd110c312c14e17a2bbe6d453493cf2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:01:13.187577  9414 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.109s (SENT)
I20260709 13:01:13.168624  8787 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:13.182757  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:13.130805 12314 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.150281 12237 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.157721 12532 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:13.204773 12546 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:01:13.182968  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:13.139856 12540 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.194653  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.183641 12436 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:13.205483  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:13.192452  8695 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:01:13.227864  8787 leader_election.cc:403] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:01:13.230981  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.144435 12433 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:13.238201  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112896us
I20260709 13:01:13.239665  8817 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:13.209460 12548 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:13.242580  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:13.169250 12533 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.231564 12283 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.302457 12283 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:13.304754 12398 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.305967  9448 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:13.306449 12398 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.307118  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.308051  9448 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:13.308898  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.309785  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.311254  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.311833  9440 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:13.312359 12233 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.312655  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.314229 12531 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.315300 12547 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:13.315840 12547 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:13.313218  9440 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.266369  8891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31c225a50048451db8964738c5db7540"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:13.320103  8891 raft_consensus.cc:606] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.320588  8891 raft_consensus.cc:614] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:13.229466 12554 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.321806  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:13.322242  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 147300us
W20260709 13:01:13.324654  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.325189  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.326556  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 139058us
W20260709 13:01:13.323120  8695 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:13.337759  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.300017 12198 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.338491  9449 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:13.319415 12548 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:13.346004  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:13.343456  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 200184us
I20260709 13:01:13.326084 12353 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.346580 12393 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:13.349208 12195 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:13.361222 12558 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:01:13.378870 12540 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.281349 12563 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.379597 12563 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:13.379482  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.380252 12563 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.380548  9442 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:13.381065 12314 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.382345 12237 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.383255 12185 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:13.384246  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.363947  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:13.347641  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:13.348071  8789 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.137s (SENT)
I20260709 13:01:13.349318 12486 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:13.349103 12555 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.193245 12049 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.375494  9444 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:13.417960  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.418272  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:13.425462  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:13.426551  8824 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.412815  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.429256  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.429840  9449 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:13.429596  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:13.430392  8817 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.430707  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.425513  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:13.431826  8891 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:13.369961 12562 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:01:13.439283  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:13.442019  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 94469.8us
I20260709 13:01:13.410847 12555 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.373505 12353 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:13.448066  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:13.438100  8788 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.098s (SENT) [suppressed 72 similar messages]
I20260709 13:01:13.447814 12555 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.454901  8815 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:13.330770 12554 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.425616 12314 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:13.468755  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 125892us
I20260709 13:01:13.445035 12185 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:13.455781 12353 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.493180  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.506858  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.508131  8894 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.508762  8891 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:13.514626  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43663.2us
I20260709 13:01:13.377633 12436 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:13.519044  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.375370 12524 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.526258  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:13.531778 12552 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.533406  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.542948  8894 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:13.544395  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 180607us
I20260709 13:01:13.550221  8891 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:13.551522  9440 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:13.552872  9441 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:13.374167 12548 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.471143 12393 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:01:13.507938 12314 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:13.386005 12547 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.567121  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:13.568970  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53399us
I20260709 13:01:13.572415  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.561254  8817 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:13.575528  9440 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:13.579463  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.575635  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:13.582027  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:13.582641  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:13.584733  8894 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:13.586011  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 403174us
I20260709 13:01:13.590194 12539 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:13.592892  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:13.554934 12185 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:13.597069  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 255290us
W20260709 13:01:13.593899  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.594300  9444 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:13.573096 12401 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:13.599819  9412 connection.cc:256] RPC call timeout handler was delayed by -1.2955826979999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 13:01:13.612442  8895 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:13.619339  8789 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.206s (SENT)
W20260709 13:01:13.622584  8863 leader_election.cc:336] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.839s (SENT)
I20260709 13:01:13.626384 12552 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.627246 12552 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:13.591678 12314 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.630414 12300 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:13.631196 12300 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:13.633641 12547 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.639684 12049 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:13.642596  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.643064 12185 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.645438 12533 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.646760 12331 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.628615 12552 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.647775 12571 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.648258 12571 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:13.648626 12571 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.648772  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:13.632411 12300 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.650035 12300 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.651721  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.652511  9443 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:13.653386  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.654418  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.655217  9440 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:13.657012  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.658108  9440 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:13.660252  8824 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:13.595443 12532 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.663672 12532 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.669466 12401 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:13.670637 12401 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:13.671428 12565 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:01:13.673607 12554 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:13.677488  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 94989.7us
I20260709 13:01:13.679064  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:13.680042  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58808.6us
I20260709 13:01:13.580518 12486 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
W20260709 13:01:13.682724  9414 leader_election.cc:336] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 3.214s (SENT)
I20260709 13:01:13.683296  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.681037  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:13.689402  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.690322  9440 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:13.650413  8788 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:13.696552  9442 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:13.703665  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.705845  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.706660 12237 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.610276 12540 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.709664 12540 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.711793 12540 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.713191 12195 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.718500  9443 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:13.602687 12436 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:13.720592  8863 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.888s (SENT)
W20260709 13:01:13.731073  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 126954us
I20260709 13:01:13.738911  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.740033  9442 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
W20260709 13:01:13.749074  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 98806us
I20260709 13:01:13.750337  8788 leader_election.cc:400] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10.
I20260709 13:01:13.750739  8788 leader_election.cc:403] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:13.755681  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.723996 12540 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:13.770211  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:13.775312  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88701.8us
I20260709 13:01:13.720041 12536 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.802600 12548 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.802469 12536 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.732506 12545 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.789846  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:13.792629  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.793771  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:13.806747  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.795393 12486 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10.
I20260709 13:01:13.796010 12571 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:13.797000  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:13.798816 12283 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:13.801551  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 122586us
I20260709 13:01:13.721354 12353 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.746109 12563 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.807237  9442 raft_consensus.cc:2376] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:13.820030  8815 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:13.820782  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.829121  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.829959  9443 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:13.832831  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.833561  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:13.728890 12555 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:13.841527  8789 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.589s (SENT)
W20260709 13:01:13.806545  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74781.9us
I20260709 13:01:13.826445  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:13.839998  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:13.847066  9440 raft_consensus.cc:2376] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:13.852033  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.855104  9449 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:13.827049 12195 raft_consensus.cc:2764] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:01:13.864921  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45889.1us
I20260709 13:01:13.854405 12439 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.848421 12237 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:13.852607 12207 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.848644 12406 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:13.879021  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:13.882329  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 75657.8us
W20260709 13:01:13.915871  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 334019us
W20260709 13:01:13.924108  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.824s (SENT)
I20260709 13:01:13.863606 12545 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.853725 12198 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:01:13.868244 12552 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.915598 12554 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.928360 12406 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.946355 12406 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.946916 12406 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:13.947228 12406 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:13.948033 12406 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.949183 12406 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.949357  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:13.949739 12532 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.950237 12532 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:13.924110 12237 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:13.953645  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:13.956183  9443 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:13.880280 12439 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:13.964891  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91687.6us
W20260709 13:01:13.965312  9414 leader_election.cc:336] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.937s (SENT)
I20260709 13:01:13.966153  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:13.940871 12195 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:13.973656  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 182681us
W20260709 13:01:13.975901  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.977890 12563 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.978546 12553 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.978744 12491 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.979187 12553 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.979337 12491 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:13.980500  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.981036  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.981441 12424 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.938761 12207 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:13.982283  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.984057  8889 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.950626 12532 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.984097  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:13.989049  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:13.983781 12563 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.983683 12424 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.992686 12563 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:13.993400  8891 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:13.994506 12424 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.994997 12563 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:13.997476  8897 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:13.998092 12439 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:13.984097 12331 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.998872  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.999897 12331 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.000348 12331 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.000514  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:13.986711 12195 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:14.001185  8892 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.001950 12195 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.002943 12185 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.003665 12185 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.000728 12331 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.008438 12553 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.009797  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:13.992715  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.011338  8893 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:14.012295  8823 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:14.014673  8892 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:14.016662 12580 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:14.017554 12580 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.017932 12580 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:14.018323 12580 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.019631 12524 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:14.020855 12331 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.022691 12433 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.023696 12314 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:14.025368 12314 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:14.026167  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.027004 12486 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:14.027318 12553 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:14.027931 12486 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:14.028445 12553 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:14.028625 12547 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:14.028914 12553 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:14.029105 12547 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:13.972487 12552 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.029838 12552 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.029974 12553 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.031584 12553 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.032500 12552 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:14.026463 12314 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:14.033105 12314 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:14.033584  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.032802  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.030004 12547 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:14.035019 12566 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.684s	user 0.003s	sys 0.000s
I20260709 13:01:14.035422 12547 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.035597 12207 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.036015  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.036835  8748 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.037745 12533 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:14.039007  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.039753  8748 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.041790 12571 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:13.993235 12582 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:01:14.043121 12423 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.043754  8897 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.043787  8740 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:14.045431 12577 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 13:01:14.045686  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.047323  8746 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.034442  8741 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:14.028475 12486 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:14.049842 12486 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.033955  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.052585  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.053041 12554 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:14.053426  8746 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.053516 12554 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.057166  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.058470  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:14.058867 12581 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:01:14.059293  8747 raft_consensus.cc:2368] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 7. Current term is 8.
I20260709 13:01:14.059432  8741 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:14.060263 12555 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:14.023983  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 88459.1us
I20260709 13:01:14.064244  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:14.035640 12314 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.077844  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:14.086547  8889 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:14.088492  8741 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:13.990527 12198 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.105866  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:14.106104  8789 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.850s (SENT)
I20260709 13:01:14.106806  8893 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:01:14.107846  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 142615us
W20260709 13:01:14.108667  9414 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.941s (SENT)
I20260709 13:01:14.109120  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:14.126628  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62512.7us
I20260709 13:01:14.077700 12207 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.036855 12536 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:14.132335  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:14.133037  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.133805  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 307495us
I20260709 13:01:14.103646 12540 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:14.138789  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.141309  8896 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.147316  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.148013  8893 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:14.148169  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:14.151688  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.154719  8898 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:14.154541  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.155468  8817 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:14.135191 12533 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.133167 12556 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:14.137017 12353 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:14.157346  8863 leader_election.cc:336] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 3.108s (SENT)
I20260709 13:01:14.159327  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.159425  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:14.160786  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.161821  8823 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:14.164224  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:14.164908  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.169180  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.169909  8893 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:14.170188  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.171216  8823 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:14.131821 12207 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:14.172914  8713 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.644s (SENT)
I20260709 13:01:14.184463  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.179730  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:14.185977  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.191936  8747 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.191979  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.163517 12314 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.193769  8817 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:14.196022  8816 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:01:14.199411 12589 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:01:14.206054 12533 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.206944 12207 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:14.144709 12237 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.207484  8741 pending_rounds.cc:85] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:01:14.207840  8741 pending_rounds.cc:107] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:01:14.208066  8741 raft_consensus.cc:2889] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:14.210844  8897 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:01:14.213430  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.213773 12049 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:14.214166 12049 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:14.214993 12049 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.159087 12556 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:14.216246 12049 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.216378 12556 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.217491 12556 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.222146 12598 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.222649 12185 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.225371 12539 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:14.231417 12539 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.231810 12539 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.232343  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 760) took 1094 ms. Trace:
I20260709 13:01:14.232739  9445 rpcz_store.cc:276] 0709 13:01:13.138186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.152657 (+ 14471us) service_pool.cc:224] Handling call
0709 13:01:13.169185 (+ 16528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.339906 (+170721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.339944 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.339985 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:13.340020 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.232106 (+892086us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400660dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:14.232149 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.232283 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392188672}
W20260709 13:01:14.234797 12559 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.967s	user 0.000s	sys 0.003s
I20260709 13:01:14.236552  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 692) took 1547 ms. Trace:
I20260709 13:01:14.239923 12491 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:14.241503 12594 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:14.242504  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 96212.5us
W20260709 13:01:14.244174  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.245625  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.246762  8818 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.246837 12185 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.247581 12185 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.249351 12353 raft_consensus.cc:2764] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:14.250115 12401 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:14.249661 12540 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.250502 12401 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.251233 12401 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.252262 12547 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.244930  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.253769 12598 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.254437 12598 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.257493 12598 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.258556  8819 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.236912  9446 rpcz_store.cc:276] 0709 13:01:12.689168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.689314 (+   146us) service_pool.cc:224] Handling call
0709 13:01:12.707095 (+ 17781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:12.781339 (+ 74244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:12.957602 (+176263us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.118355 (+160753us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.636024 (+517669us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b2c000ebf38. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:13.636518 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.636660 (+   142us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:13.636696 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.636771 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.667409 (+ 30638us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b180034a200. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90fa3a358f 00007f90f33e2571 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd
0709 13:01:14.234140 (+566731us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:14.236355 (+  2215us) raft_consensus.cc:1685] finished
0709 13:01:14.236376 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.236496 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25230,"prepare.queue_time_us":356944,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":42456,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":12799,"spinlock_wait_cycles":91449088,"thread_start_us":263963,"threads_started":2,"wal-append.queue_time_us":156342}
I20260709 13:01:14.193308 12536 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:14.263126  8862 leader_election.cc:336] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.064s (SENT)
W20260709 13:01:14.248986 12436 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.521s	user 0.004s	sys 0.003s
I20260709 13:01:14.267410  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.270224  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.283740  8823 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:14.277511  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 89590.3us
I20260709 13:01:14.285616  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.286245  8821 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.278458  8824 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:14.281563  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.287293  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:14.288757  8898 raft_consensus.cc:606] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:01:14.289211  8898 raft_consensus.cc:614] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:14.286437  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.297453  8822 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:14.287499  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:14.297657  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 146077us
I20260709 13:01:14.252789 12547 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:14.300072  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 149019us
I20260709 13:01:14.305032  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.314144  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.314917  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:14.315785  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:14.317417  8819 raft_consensus.cc:606] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:14.317756  8819 raft_consensus.cc:614] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:14.320987  8747 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:14.321303  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.322324  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.322435  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.323457  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.323693  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:14.324146  8747 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.324890  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.328266  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.313784 12554 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.329633  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.330015  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.330705  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.330947  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:14.331355  8746 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:14.331562 12554 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.332267  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.332686  8822 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.332831  8746 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:14.333999  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.334745  8744 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:14.335071  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 162219us
I20260709 13:01:14.323462  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.337420  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:14.340873  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.341609  8747 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:14.341857  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54444us
I20260709 13:01:14.315212  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:14.340633  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:14.345695  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.342499  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.347704  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.347738  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.350411  8743 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:14.351771  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46929.3us
I20260709 13:01:14.352934  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:14.355373  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.360780  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.364542  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.295161 12393 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:01:14.366536  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.341199 12486 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:14.371896  8747 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.369936  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.373116  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:14.376998  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.380750  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.378268  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.368952  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:14.391472  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 128404us
I20260709 13:01:14.393868  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.395145  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:14.399348  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.400904  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.402468  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.403352  8889 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:14.361821 12331 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:14.367355 12531 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.403514  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.404753  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:14.408757 12597 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.408954  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.425266  8747 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.409437 12533 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:01:14.367449 12237 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:14.416990  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:14.412482  8749 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:14.383628 12594 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:14.426707  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:14.430977  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62129.2us
W20260709 13:01:14.432925  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.438846  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.425092 12597 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:14.439745 12597 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.405071 12491 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.400411 12486 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:14.378118 12542 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:14.441561  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.433115  8898 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:14.442301  9446 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:14.376595 12433 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:01:14.431901 12237 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.435788 12594 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.436553 12439 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:14.474462  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:14.474892  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 74273.1us
I20260709 13:01:14.489219  9445 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.470474  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.443269 12491 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.486960  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.492578  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.493438  8889 raft_consensus.cc:2368] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:01:14.450389 12185 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:14.512017  8713 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.753s (SENT)
I20260709 13:01:14.512789  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:14.518762  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.437041 12531 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.525923  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.527616  9446 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:14.478163 12237 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.474748 12552 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:14.593971 12552 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.594589 12552 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.529250  8898 raft_consensus.cc:2368] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
W20260709 13:01:14.530122  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.531213  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.530890  8821 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:14.493795 12439 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.600826 12439 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.467324 12542 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.603922  8815 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:14.599306  8746 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.613605  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:14.506238 12491 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.532594  8816 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:14.531199  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.489805 12594 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.532081 12576 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.529179 12599 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.242s
I20260709 13:01:14.642712 12599 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:14.643169 12599 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:14.644163 12599 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.539655  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.521487 12423 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
W20260709 13:01:14.545004 12553 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.003s
I20260709 13:01:14.548357 12547 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:14.548813 12595 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:14.550055 12610 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:01:14.554839  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.653338  9444 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.556501  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.558113  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:14.564838 12614 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.079s	user 0.002s	sys 0.001s
I20260709 13:01:14.565766  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.567384  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.567214  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.568578  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:14.568884 12601 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:01:14.569415  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.571621  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:14.574584 12615 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.090s	user 0.002s	sys 0.000s
I20260709 13:01:14.577991  8742 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
W20260709 13:01:14.586498 12580 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.568s	user 0.005s	sys 0.006s
W20260709 13:01:14.588884 12545 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.650s	user 0.002s	sys 0.008s
W20260709 13:01:14.589795 12593 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:01:14.599218  8713 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.098s (SENT)
I20260709 13:01:14.638600  8821 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:14.646450  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:14.645658  8741 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:14.528978 12571 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.647910  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.682194  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:14.636113 12406 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.005s
W20260709 13:01:14.653687  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.649519 12554 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:14.656139 12595 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:14.658429  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.634016 12195 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.693188  8889 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:14.662139  9442 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:14.662156  9448 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:14.662457  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.662884  8744 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.660254  9447 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
I20260709 13:01:14.664461  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.695442  8742 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.666882  8740 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:14.664402  8745 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:14.697542  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.680558  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:14.677057 12532 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260709 13:01:14.702499  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.686303  8898 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:14.688495  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:14.688890  8893 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:14.689736  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.699939  9440 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:14.707738  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:14.709556  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.709975  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.710754  8891 raft_consensus.cc:2368] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:01:14.715538  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.709429  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.729521  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:14.731052  8747 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:14.732944  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.734467  8894 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:14.735122  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.735644  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.735895  8889 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:14.736249  8742 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.736852  8746 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:14.737203  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.684799 12545 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.687302 12207 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.740574  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.741376  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.742832  8712 leader_election.cc:400] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:01:14.743340  8712 leader_election.cc:403] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:01:14.743947  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:14.744459  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.744638  8893 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:14.745464  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.745725  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.746073  9447 raft_consensus.cc:2376] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11. Re-sending same reply.
I20260709 13:01:14.746501  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:14.747063  9442 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.738302  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.667812 12580 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.659540 12486 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.665256 12331 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:01:14.749269  9446 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.751310  8749 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:14.756906  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.757529  8746 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.760010  8820 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:14.761370  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.762490  8893 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:14.767845  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:14.768623  9444 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:14.713219 12314 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.770385  8892 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:14.723356 12353 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.772400  9448 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:14.830101  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:14.822429  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.817574  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.667227 12398 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:01:14.833950  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:14.835846  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.842077  9447 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:14.831594  8741 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
W20260709 13:01:14.843408  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 244279us
I20260709 13:01:14.737739 12531 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:14.848099  9447 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:14.857400  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.723630 12532 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:14.862998  8746 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:14.870896  8747 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:14.882053  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 139368us
W20260709 13:01:14.884675  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.886375  8742 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:14.845132 12545 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:14.747671 12578 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:14.887117  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.895430  8746 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.887562  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.886482  9442 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:14.887598  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.898947  8897 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:14.750099 12604 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:01:14.744720 12571 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.911931  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:14.912504  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.912906  8742 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:14.893441  8712 leader_election.cc:400] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:14.914300  8712 leader_election.cc:403] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:14.748245 12207 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:14.856413 12442 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.916558  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.918982 12207 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.919931  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.752447 12580 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:14.920740  8744 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.924232  8747 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:14.850692 12542 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:14.918079  9447 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.948678  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:14.948933  9447 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.954280  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.958384  8893 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 7: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:14.966248  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.913697 12545 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.987996  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.989491  8741 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:14.999740  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44216us
W20260709 13:01:14.969270  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 239927us
W20260709 13:01:14.983793  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.984535  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:14.984645  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:14.985427  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 258888us
I20260709 13:01:15.006095  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:15.007015  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.002401  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.024482  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:14.952723 12314 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:14.998323 12545 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.921489 12571 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.031034  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.035673  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.036895  8746 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:15.039472  8741 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:15.040743  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:15.047326 12442 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9.
I20260709 13:01:15.047381  8890 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:15.053004  8744 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:15.053929  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.054590  8741 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.055635  8742 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:15.062976 12532 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.063529 12439 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:14.895637 12595 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.066080 12439 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:15.067272  8741 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:15.068360  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.069597  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.070684  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.071308 12571 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.071812  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.072443  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:15.073675  8741 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:15.074951  8741 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:15.076601  8747 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
W20260709 13:01:15.094244 12539 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.012s
I20260709 13:01:15.094971 12539 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.121407  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:15.031889 12623 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.136916  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.152066  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 556) took 1111 ms. Trace:
I20260709 13:01:15.158592  8893 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.029249 12613 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:15.010094 12580 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.099s
W20260709 13:01:15.172676  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 51420.9us
I20260709 13:01:15.153026  8748 rpcz_store.cc:276] 0709 13:01:14.040603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.040723 (+   120us) service_pool.cc:224] Handling call
0709 13:01:14.041062 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.041549 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.041577 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:14.042087 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:01:14.158302 (+116215us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:14.158351 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.158428 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:15.151424 (+992996us) raft_consensus.cc:1685] finished
0709 13:01:15.151622 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.151944 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":944,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":115344,"threads_started":1,"wal-append.queue_time_us":404838}
I20260709 13:01:15.176407 12624 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.188724  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:15.032236 12542 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:01:15.195020  8893 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:15.172062 12571 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.202265  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.206979  8746 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:15.204041  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:15.198558  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.210014  9447 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:01:15.217599  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.229863 12621 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:01:15.230104  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.231040  8748 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:15.231940  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.232635 12627 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.207165 12624 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:15.235236 12624 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.238168  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.238916  8746 raft_consensus.cc:2376] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7. Re-sending same reply.
I20260709 13:01:15.257072  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.258494  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.259181  8746 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:15.233271  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.262467  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:15.266321  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:15.268111  8863 leader_election.cc:336] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.520s (SENT)
I20260709 13:01:15.269186  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:15.271961  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:15.275170  8897 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
W20260709 13:01:15.279601  8789 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.193s (SENT)
I20260709 13:01:15.280120  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.282527  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:15.182307 12613 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.294713  9442 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:15.284688  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.290102  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:15.290772  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:15.285665  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 252527us
I20260709 13:01:15.296038  9447 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:15.295739  8748 raft_consensus.cc:2376] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
I20260709 13:01:15.296809  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:15.205802 12580 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:01:15.290109  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.307936  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.308777  8747 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
I20260709 13:01:15.271715 12539 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.263142 12571 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.307112  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:15.318251  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.324054  8748 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:15.314894 12554 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:15.325028  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53189.4us
I20260709 13:01:15.326069  8815 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:15.327049  8894 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.315146 12588 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:15.328670 12588 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.331723 12588 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:15.334280  8815 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.335873  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:15.342490  8821 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:15.347635 12207 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:15.349275 12552 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.754s	user 0.007s	sys 0.007s
W20260709 13:01:15.350594 12401 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.099s	user 0.013s	sys 0.000s
I20260709 13:01:15.306138 12542 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.352804  8746 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:15.355477  8822 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.356065  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:15.370980  8816 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:15.356452  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:15.358986  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 105123us
W20260709 13:01:15.271188 12540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.013s
I20260709 13:01:15.381713  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.383972  8746 raft_consensus.cc:2376] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7. Re-sending same reply.
I20260709 13:01:15.384729  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.385228  8746 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:15.382002  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.361855  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.386137  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.310479 12627 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.366359  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.367653  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:15.376642  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.369148 12401 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.378420 12596 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.379616  8892 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:15.380841  8819 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:15.363020  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.386754  8821 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:15.393191  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:15.396270  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:15.396174  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 393930us
W20260709 13:01:15.382262 12540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.007s	sys 0.013s
I20260709 13:01:15.397766  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.416500  8816 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:15.400197  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.418927  8746 raft_consensus.cc:2376] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8. Re-sending same reply.
I20260709 13:01:15.420212  8746 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:15.422164  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.423243  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.425596  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:15.429560  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62043.6us
I20260709 13:01:15.429744  8816 raft_consensus.cc:2368] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:15.431026  8824 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.431795  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.432710  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:15.434643  8894 raft_consensus.cc:2368] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:15.435441  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:15.363190 12552 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.407931 12598 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:15.445784  8746 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:15.446245  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.415295 12542 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.440284  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.438550  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.415732 12406 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.456084  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:15.457104  8748 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:01:15.461685  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:15.462980 12532 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.463438 12532 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.464443 12532 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:15.465878 12630 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:15.466239  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 584) took 1225 ms. Trace:
I20260709 13:01:15.487053  8745 rpcz_store.cc:276] 0709 13:01:14.240527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.337179 (+ 96652us) service_pool.cc:224] Handling call
0709 13:01:15.466038 (+1128859us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400161054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.466178 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587820288}
I20260709 13:01:15.466949 12401 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:15.467387  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 558) took 1422 ms (client timeout 1824 ms). Trace:
I20260709 13:01:15.468125  8743 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:15.470687  8742 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:15.476039  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 1983) took 1453 ms. Trace:
I20260709 13:01:15.476958  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1642) took 1102 ms. Trace:
I20260709 13:01:15.479903  8744 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:15.452294 12613 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.449572 12622 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:15.454604 12595 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:15.484513 12633 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.441622 12539 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.517468  8745 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:15.499554  8746 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:15.507840  8749 rpcz_store.cc:276] 0709 13:01:14.374981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.440518 (+ 65537us) service_pool.cc:224] Handling call
0709 13:01:15.476772 (+1036254us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400466ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.476883 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599866624}
I20260709 13:01:15.519387  8745 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:15.509490  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 74143.4us
I20260709 13:01:15.520887  8745 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.405179 12627 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.524065  8746 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.513309  8890 rpcz_store.cc:276] 0709 13:01:14.022912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.033726 (+ 10814us) service_pool.cc:224] Handling call
0709 13:01:15.475814 (+1442088us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b540004b5d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.475971 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469900032}
W20260709 13:01:15.514214  8740 rpcz_store.cc:269] 0709 13:01:14.045325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.045445 (+   120us) service_pool.cc:224] Handling call
0709 13:01:15.467228 (+1421783us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b540013c4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.467332 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427424000}
I20260709 13:01:15.509479  8747 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:15.532505  8742 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.442680 12442 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:01:15.542568  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:15.543516  8889 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:15.522300  8745 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.545605  8745 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.547073  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:15.555629  8742 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:15.556140 12552 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:15.570880 12198 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:15.571568 12198 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:15.574239 12639 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:15.578496 12314 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.579057 12314 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.580602 12623 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:15.581084 12623 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:15.582262 12623 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.584398 12623 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.586174  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2254) took 1342 ms. Trace:
I20260709 13:01:15.586704  8820 rpcz_store.cc:276] 0709 13:01:14.244013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.244662 (+   649us) service_pool.cc:224] Handling call
0709 13:01:15.585901 (+1341239us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54003e35d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.586056 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240127872}
W20260709 13:01:15.595105  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76247.3us
W20260709 13:01:15.602147 12629 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:01:15.427856 12596 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.517592 12588 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:15.606854  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.607366  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 180668us
I20260709 13:01:15.611279  8744 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:15.614233  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.523396 12406 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.615864  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.618996  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.584849 12207 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:15.626047  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 202905us
I20260709 13:01:15.629876  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:15.613095 12594 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.963s	user 0.009s	sys 0.006s
W20260709 13:01:15.627552  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 106876us
I20260709 13:01:15.627890  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:15.620728 12607 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.532841 12539 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.504894 12571 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:15.643589  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.611361 12624 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:15.626998 12588 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:15.647621  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.624933 12580 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:15.633030 12442 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.650992 12598 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:15.652892  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:15.654208  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.654429  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:15.648367 12207 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.636545 12594 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.655290  8745 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 4
I20260709 13:01:15.656153  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.675144  8745 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:15.675426  8745 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.673872  9445 raft_consensus.cc:2368] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9.
I20260709 13:01:15.643869 12532 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.656136  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.704489 12571 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.706856 12442 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.708065 12442 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.710224 12598 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.663012 12627 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:15.713047  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56972.3us
I20260709 13:01:15.713948  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:15.715687  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.663101 12207 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.667078 12594 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:15.658596 12596 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:15.717784 12596 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.718564 12596 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.719108 12596 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:15.720203 12596 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.721580 12423 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:15.669081 12597 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.208s	user 0.008s	sys 0.003s
I20260709 13:01:15.724071 12597 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.729526  8897 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:15.733645 12331 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.734431 12331 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.663576 12588 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.741842  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:15.744731  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:15.658135 12552 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.752856  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:15.648509 12536 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:15.654709 12393 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:15.759479  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103444us
I20260709 13:01:15.771811  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:15.772101  8788 connection.cc:256] RPC call timeout handler was delayed by -1.0806840139999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:01:15.772905  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.769181  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:15.665947 12198 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:15.776566  9447 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.642817 12607 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:15.760387 12545 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.001s
I20260709 13:01:15.782258  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:15.780984  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.664845 12539 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:15.771538 12554 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.822062 12554 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.824414 12554 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.792403 12207 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:15.793514 12647 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:15.794268 12646 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:15.827299  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46423.6us
I20260709 13:01:15.829232  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 582) took 1589 ms. Trace:
I20260709 13:01:15.831488  8743 rpcz_store.cc:276] 0709 13:01:14.239880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.366681 (+126801us) service_pool.cc:224] Handling call
0709 13:01:15.467758 (+1101077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.512388 (+ 44630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.829040 (+316652us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400161050. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:15.829173 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476924160}
I20260709 13:01:15.832155 12643 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.832598 12643 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.832947 12643 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.839526  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:15.842382  8742 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:15.790748 12237 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.152s	user 0.016s	sys 0.005s
I20260709 13:01:15.774087 12486 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.837762 12641 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.833815 12331 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:15.834038 12198 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:15.848662  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:15.850143  8748 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.853534  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.836297 12595 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:01:15.841271 12545 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.854414 12641 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.854673 12486 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:15.849911 12237 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.865192  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:15.877526  8895 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:15.842790 12233 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.867420 12597 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:15.865034 12198 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:15.845484 12651 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:15.773473 12536 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.833088 12433 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:01:15.846686 12539 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.865679  8748 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:15.901645  9447 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:15.872033 12353 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.022s	user 0.008s	sys 0.005s
I20260709 13:01:15.845781 12622 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:01:15.845646 12594 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:01:15.917560  8893 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:15.922492  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.935952  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 164274us
I20260709 13:01:15.939323  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.907829 12540 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.940006  8820 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:15.904436 12552 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:15.928791  8748 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.922091 12635 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.928889 12353 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.956346  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.963064  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:15.965721  8822 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.963836  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:15.964915  8748 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.944167 12540 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:15.970713  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.976342  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.971117  8819 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:15.971009  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:15.971272  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 343495us
W20260709 13:01:15.959841 12491 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.312s	user 0.008s	sys 0.007s
I20260709 13:01:15.978137  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:15.921514 12539 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.980230  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.982191  8715 leader_election.cc:400] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:15.982532  8715 leader_election.cc:403] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.952502 12613 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:15.983451  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:15.978271 12540 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.914361 12651 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.994747  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:15.994549  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:15.989513 12442 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:15.987191 12654 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:16.014230  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:16.032009 12622 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:16.036522 12622 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:16.037030 12622 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.038038 12607 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:16.045639 12597 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.046154 12597 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:16.045833 12622 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.047021 12597 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:15.992992 12655 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:15.967963 12635 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.981349 12491 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.990294 12637 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.187s
I20260709 13:01:15.979697 12486 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:15.988919 12198 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.001130 12595 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:01:15.993261 12613 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:16.181763  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.418038  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 403937us
W20260709 13:01:16.418444  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.121680  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 707) took 1557 ms. Trace:
I20260709 13:01:16.157620  8748 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:16.013705 12442 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.419507  9443 rpcz_store.cc:276] 0709 13:01:14.564738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.564879 (+   141us) service_pool.cc:224] Handling call
0709 13:01:14.565226 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.649617 (+ 84391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.649650 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:14.655577 (+  5927us) log.cc:844] Serialized 26 byte log entry
0709 13:01:15.605710 (+950133us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b18003ff340. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:01:15.932086 (+326376us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:15.932133 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.933234 (+  1101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.121345 (+188111us) raft_consensus.cc:1685] finished
0709 13:01:16.121369 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.121614 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82263,"raft.queue_time_us":86439,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3609,"spinlock_wait_cycles":9291008,"thread_start_us":326158,"threads_started":1,"wal-append.queue_time_us":1329427}
I20260709 13:01:16.426973  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2223) took 1062 ms. Trace:
I20260709 13:01:16.452903  8897 rpcz_store.cc:276] 0709 13:01:15.364301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.364904 (+   603us) service_pool.cc:224] Handling call
0709 13:01:15.365314 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.730102 (+364788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.426723 (+696621us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400248454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:16.426911 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837227776}
W20260709 13:01:16.428709  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.292850  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2219) took 1083 ms. Trace:
W20260709 13:01:16.390655  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.287279  9445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 701) took 1791 ms (client timeout 1601 ms). Trace:
I20260709 13:01:16.417178 12599 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.124336 12393 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.698770  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:16.505421  8893 rpcz_store.cc:276] 0709 13:01:15.209395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.214625 (+  5230us) service_pool.cc:224] Handling call
0709 13:01:15.215058 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.925094 (+710036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.292496 (+367402us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54000947d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:16.292770 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1615173888}
W20260709 13:01:16.506208  9445 rpcz_store.cc:269] 0709 13:01:14.495903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.496312 (+   409us) service_pool.cc:224] Handling call
0709 13:01:16.286916 (+1790604us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:16.287125 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388629248}
W20260709 13:01:16.518143  8713 leader_election.cc:336] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.527s (SENT)
I20260709 13:01:16.032291 12433 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.529759 12433 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.531579 12491 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.087010 12552 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:16.533145  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:16.534062  8897 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:16.065275 12618 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.182s	user 0.004s	sys 0.000s
I20260709 13:01:16.235066 12237 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:16.450428 12398 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:01:16.234735 12571 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:16.527139 12626 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.528257  8895 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:16.531836 12433 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:16.532487 12657 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.570s	user 0.004s	sys 0.000s
I20260709 13:01:16.447664  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 745) took 1202 ms. Trace:
I20260709 13:01:16.534734  9444 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:16.603677  8822 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.006s
I20260709 13:01:16.069476 12536 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.064853 12651 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:16.401250 12331 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:16.121788  8789 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.500s (SENT)
W20260709 13:01:16.607961  8715 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.031s	sys 0.056s
I20260709 13:01:16.618849  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:16.619261  8715 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.032s	sys 0.057s
W20260709 13:01:16.536878  8712 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.034s	sys 0.047s
W20260709 13:01:16.340346 12660 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:01:16.546533  8897 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:16.537245  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:16.552712  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.000s	sys 0.016s
W20260709 13:01:16.551667  8820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.011s
W20260709 13:01:16.622133  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.000s	sys 0.016s
W20260709 13:01:16.622223  8820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.000s	sys 0.012s
I20260709 13:01:16.588248  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:16.225709 12545 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:16.465792 12556 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:01:16.592110  9446 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:16.070770 12648 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.568s
I20260709 13:01:16.476382 12597 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.465721 12595 consensus_queue.cc:1243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:16.601387  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:16.523036  8863 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.728s (SENT)
I20260709 13:01:16.631335  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:16.631791  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:16.632194  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 109301us
I20260709 13:01:16.632624  8895 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.604200  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:16.465821 12442 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.466387 12424 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:16.643062  9441 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:16.353790 12406 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:16.589576 12623 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.004s	user 0.016s	sys 0.000s
W20260709 13:01:16.612181  9440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 747) took 1362 ms (client timeout 1527 ms). Trace:
I20260709 13:01:16.613914  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:16.564981 12622 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.071463 12654 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:16.619588  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 636276us
I20260709 13:01:16.584625 12198 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:16.619752  8712 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.037s	sys 0.047s
I20260709 13:01:16.531749 12607 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:16.551803 12641 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.589850 12658 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.624471  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:16.625967  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 504286us
I20260709 13:01:16.602694  9442 rpcz_store.cc:276] 0709 13:01:15.244909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.247504 (+  2595us) service_pool.cc:224] Handling call
0709 13:01:15.247854 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:15.295166 (+ 47312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.289755 (+994589us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b540050f854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:16.447560 (+157805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107307392}
I20260709 13:01:16.643504  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:16.450688 12655 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:16.659909  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.657681  9440 rpcz_store.cc:269] 0709 13:01:15.249264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.249416 (+   152us) service_pool.cc:224] Handling call
0709 13:01:16.611965 (+1362549us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400660dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:16.612105 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603191680}
W20260709 13:01:16.681883  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 48520.9us
I20260709 13:01:16.618693 12331 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:16.736212  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:16.736485 12331 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.737473  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:16.738009 12331 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:16.738368 12553 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:16.739848 12553 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.740574 12553 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.622723 12539 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:16.692878  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 704974us
I20260709 13:01:16.742904  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:16.693331  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.743825  8748 raft_consensus.cc:2376] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7. Re-sending same reply.
I20260709 13:01:16.744432  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:16.627416 12651 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:16.693748  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.692412  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:16.746064 12393 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.689790  9448 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:16.258733 12314 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.697386  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.747848  8746 raft_consensus.cc:2376] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11. Re-sending same reply.
I20260709 13:01:16.699357  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:16.749634  8742 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:16.636435 12536 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.724983  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:16.725004  8822 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:16.751132  8897 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.728232  8740 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:16.623095 12666 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:16.642930 12433 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:16.753260  8743 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:16.761816  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:16.761901  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:16.762600  8819 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:16.764664 12596 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.765302 12596 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.767992 12596 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:16.773274 12539 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:16.778084  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.607187 12626 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.748219  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:16.724331 12641 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:16.802452 12672 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.803153 12672 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:16.807838  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:16.790063 12580 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.023s	sys 0.040s
I20260709 13:01:16.699918 12623 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:01:16.792001  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:16.792387 12666 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.792518 12680 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.797070  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2257) took 2543 ms. Trace:
I20260709 13:01:16.793344 12665 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:16.799211  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 343833us
W20260709 13:01:16.790838 12661 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.455s	user 0.000s	sys 0.001s
W20260709 13:01:16.800629  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 52484us
I20260709 13:01:16.689049 12406 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.688633 12424 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:16.790390  8824 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:16.826236 12580 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.023s	sys 0.041s
I20260709 13:01:16.813541 12486 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.730801 12607 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:16.835547  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.853237  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.854954  8892 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:16.846745  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:16.830421  8818 rpcz_store.cc:276] 0709 13:01:14.247834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.281372 (+ 33538us) service_pool.cc:224] Handling call
0709 13:01:14.281739 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.281987 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.282016 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:14.297178 (+ 15162us) log.cc:844] Serialized 26 byte log entry
0709 13:01:14.665663 (+368485us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:14.665699 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.665765 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.064689 (+1398924us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:16.730669 (+665980us) raft_consensus.cc:1685] finished
0709 13:01:16.730698 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.791424 (+ 60726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15072000,"thread_start_us":331596,"threads_started":1,"wal-append.queue_time_us":405174}
W20260709 13:01:16.857388  8712 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.684s (SENT)
I20260709 13:01:16.857963  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:16.823923 12671 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.849133 12233 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:01:16.862699  8890 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.835151 12665 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:16.841722 12536 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:16.874233  9447 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:16.854251 12616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.002s	sys 0.009s
I20260709 13:01:16.777855 12658 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:16.881042  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91986.1us
I20260709 13:01:16.871050 12233 consensus_queue.cc:1243] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:16.854010 12580 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:16.830345 12641 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.859645 12596 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:16.894052  9447 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:16.877599 12616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.002s	sys 0.010s
I20260709 13:01:16.900682  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:16.903096 12682 raft_consensus.cc:2764] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:01:16.832973 12666 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.891067 12651 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:16.907363 12580 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:16.908300 12651 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:16.910091  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:16.911196 12664 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:01:16.913851 12616 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:16.914660 12424 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:16.916810 12672 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.917167 12626 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.878499 12635 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.918232 12687 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:16.919811 12655 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.924233 12672 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.924817 12672 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:16.926898 12672 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:16.927876  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:16.928772  8743 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:16.929472 12613 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:16.941068  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.942447  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.942991  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.943388  8743 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:16.943678  8890 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:16.944020  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:16.945623  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:16.946358  8742 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:16.842351 12680 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:16.941982  8746 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11. Re-sending same reply.
I20260709 13:01:16.944198  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.994581  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.995682  8746 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 7: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:16.978180  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:17.007910  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.008791  8742 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11. Re-sending same reply.
I20260709 13:01:17.010718  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:16.870500 12671 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.035962  8743 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:16.884491 12536 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:16.994251 12661 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:01:17.034569 12424 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:01:17.038619 12674 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.007335 12613 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:17.039145 12491 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:01:17.063544  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.007150 12682 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.089062  8712 leader_election.cc:400] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:01:17.089522  8712 leader_election.cc:403] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:01:17.039960  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.079160  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:17.083662  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 200955us
I20260709 13:01:17.090924  9440 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:17.091183  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:17.045131 12688 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.034744 12433 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:17.086728  8742 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:16.931128 12596 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.031299 12627 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:17.077653 12685 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:17.031718 12654 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.038622 12641 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.104684  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.105150  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 594) took 1268 ms. Trace:
I20260709 13:01:17.132310  9442 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:17.132265  8745 rpcz_store.cc:276] 0709 13:01:15.836581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.844637 (+  8056us) service_pool.cc:224] Handling call
0709 13:01:15.844962 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:15.845299 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:15.845336 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:15.866016 (+ 20680us) log.cc:844] Serialized 26 byte log entry
0709 13:01:15.991240 (+125224us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:15.999538 (+  8298us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.232829 (+233291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.232950 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.104919 (+871969us) raft_consensus.cc:1685] finished
0709 13:01:17.104939 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.105090 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110636,"prepare.queue_time_us":8100,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":696579,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":7953,"thread_start_us":270156,"threads_started":2,"wal-append.queue_time_us":294245}
I20260709 13:01:17.106168  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.044405 12616 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:17.138236  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47184.1us
W20260709 13:01:17.158543  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.084107  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.135033  8747 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:16.996708 12665 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.170300  9442 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:17.161131 12672 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8.
I20260709 13:01:17.165239 12641 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:17.177268 12641 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.177527  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.177794 12641 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:17.181753  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 92857.8us
I20260709 13:01:17.183169  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:17.184160 12675 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:17.186738 12596 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:17.188230 12686 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:01:17.176659  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:17.110061  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 263375us
I20260709 13:01:17.189857  9440 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:17.115226 12406 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:17.191157  8788 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.601s (SENT)
I20260709 13:01:17.191416 12607 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.191552 12667 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:17.192271 12667 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.193452 12671 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.194696 12667 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.137948 12623 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.197099  9442 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:17.120499 12682 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.201465  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.206290 12666 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.191634  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:17.216041  8895 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 13:01:17.262614  8895 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:17.217367  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.135994 12433 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:17.089205 12613 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:17.237358  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.339939  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 156976us
I20260709 13:01:17.316705  9440 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:17.341128  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:17.159206 12693 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.210989 12599 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.004s
I20260709 13:01:17.368172  8895 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.173640 12654 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.086301 12674 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.369555  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:17.373832  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 156625us
I20260709 13:01:17.137338 12627 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.114464 12688 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.331698 12491 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.374498  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:17.374789  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:17.387799  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 196681us
I20260709 13:01:17.384158  8889 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:17.402870  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:17.404045  8889 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:17.376405 12198 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 13:01:17.249359 12623 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.415802 12623 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:17.383327 12393 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.416240 12658 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.416659 12393 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.420692 12393 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.421837 12491 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.422255 12433 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.424263 12491 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:17.425637 12552 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.426244 12552 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.428097  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 601) took 1541 ms. Trace:
I20260709 13:01:17.428570 12552 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.429687 12690 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.431005  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.431922  9441 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:17.405227  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:17.439128  8892 raft_consensus.cc:2376] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8. Re-sending same reply.
I20260709 13:01:17.408012  8896 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:17.428508  8744 rpcz_store.cc:276] 0709 13:01:15.886582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.902309 (+ 15727us) service_pool.cc:224] Handling call
0709 13:01:15.916510 (+ 14201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.035737 (+119227us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:16.524002 (+488265us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.036683 (+512681us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.040513 (+  3830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.040577 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:17.040635 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.427844 (+387209us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400463cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:17.427891 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.428032 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69766,"prepare.queue_time_us":163471,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":2020,"raft.queue_time_us":142689,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7862,"spinlock_wait_cycles":259682560,"thread_start_us":872646,"threads_started":3,"wal-append.queue_time_us":348351}
I20260709 13:01:17.390513 12667 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.398641  8893 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:17.218312 12553 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 13:01:17.412804 12599 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:17.141886 12685 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.479461 12685 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.433133  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:17.480738  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 47710.2us
I20260709 13:01:17.482321 12553 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:17.482810 12613 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:17.483438  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.483364 12616 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:17.484241  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.434854  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 93869.5us
I20260709 13:01:17.486336  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.488766  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.370707 12671 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.437050  9442 pending_rounds.cc:85] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 8
I20260709 13:01:17.491762  9442 pending_rounds.cc:107] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:17.492069  9442 raft_consensus.cc:2889] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:17.331332 12486 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 13:01:17.465446  9445 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:17.498152  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.499173 12680 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:17.495494 12685 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:17.504317 12643 consensus_meta.cc:287] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.648s	user 0.005s	sys 0.007s
I20260709 13:01:17.510649 12627 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.512038 12536 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.517468 12671 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.520366 12671 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.522119 12696 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.522737 12696 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.524957 12696 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.374034 12698 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:17.490139  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.388943 12654 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:17.542712  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.464283 12651 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:17.472610 12532 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.731s	user 0.014s	sys 0.000s
I20260709 13:01:17.373571 12693 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.494158  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:17.537591  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 47535.2us
I20260709 13:01:17.556239  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.538925 12685 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.406894  8895 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:01:17.551719  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.348778 12682 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.482834 12553 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:17.493220 12648 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.561831  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.583104  8823 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:17.579267 12198 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.590211  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:17.598035  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.599103  9444 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:17.615563  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.616171  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:17.616559  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.551429 12651 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:17.566205 12540 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.563s	user 0.007s	sys 0.007s
I20260709 13:01:17.628741  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:17.625195  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 63525.2us
I20260709 13:01:17.626578  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 1994) took 1173 ms. Trace:
I20260709 13:01:17.630100  8712 leader_election.cc:400] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:17.630527  8712 leader_election.cc:403] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:17.629830  8894 rpcz_store.cc:276] 0709 13:01:16.453546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.453739 (+   193us) service_pool.cc:224] Handling call
0709 13:01:16.526708 (+ 72969us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:16.774281 (+247573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.774311 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:17.430479 (+656168us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:17.431147 (+   668us) log.cc:844] Serialized 52 byte log entry
0709 13:01:17.586815 (+155668us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:17.587105 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.588709 (+  1604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.588811 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:17.588867 (+    56us) raft_consensus.cc:1685] finished
0709 13:01:17.626276 (+ 37409us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.626498 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28582,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":77081,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":13239,"spinlock_wait_cycles":33084544,"thread_start_us":402340,"threads_started":2,"wal-append.queue_time_us":153467}
I20260709 13:01:17.562978 12580 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:17.637323  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.587136 12198 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.636633  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:17.645769  8823 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.643987  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:17.637964  9446 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:17.618387 12658 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.549521 12643 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.627667 12648 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:17.651505  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.652287  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.627509 12651 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.569727 12698 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:01:17.621642 12532 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:01:17.618921 12685 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:01:17.649058  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:17.672127  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71743us
I20260709 13:01:17.703648  8819 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:17.671916  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:17.705902  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 76010.4us
I20260709 13:01:17.706341  8822 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:17.707463  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:17.707850  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 79189.4us
I20260709 13:01:17.679677  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:17.707674  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.708369  8817 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:17.625945 12666 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.707716  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:17.708933  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 276848us
I20260709 13:01:17.717180  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.721697  9444 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:17.722822  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.642395 12671 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.725569  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.727628  9444 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:17.727566  8820 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:17.731602 12682 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.732257 12682 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.734054  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.734803  9441 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:17.735471  9448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 718) took 1960 ms (client timeout 2083 ms). Trace:
W20260709 13:01:17.735908  9448 rpcz_store.cc:269] 0709 13:01:15.775010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.781914 (+  6904us) service_pool.cc:224] Handling call
0709 13:01:17.735288 (+1953374us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54006720d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:17.735420 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067340160}
I20260709 13:01:17.737879 12654 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.738684  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:17.741024 12674 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.742743 12198 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:17.744848 12442 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.064s	user 0.009s	sys 0.003s
I20260709 13:01:17.745816 12685 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:17.746116 12693 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.746909 12688 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.747144  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:17.714119 12597 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.070s	user 0.009s	sys 0.004s
I20260709 13:01:17.747915 12556 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.737748  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.757051  8748 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:17.757509  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1685) took 1058 ms. Trace:
I20260709 13:01:17.758270  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:17.757946  8748 rpcz_store.cc:276] 0709 13:01:16.699561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.744879 (+ 45318us) service_pool.cc:224] Handling call
0709 13:01:17.737579 (+992700us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:17.757449 (+ 19870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100729600}
I20260709 13:01:17.763876  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.751444  8897 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:01:17.673197 12658 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.714046 12648 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.778838 12688 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:17.779238 12648 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.726466 12706 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:01:17.780635 12706 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:17.688902 12580 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:17.755867 12442 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:17.783964 12703 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.447s	user 0.001s	sys 0.000s
I20260709 13:01:17.784508 12442 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.784590 12658 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.785408 12713 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:17.786579 12713 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.787060 12713 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.787458  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.751997 12685 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:17.790333 12685 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 7, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.791671 12685 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.792347  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.793437 12536 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:17.793965 12536 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.794318  8816 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:17.795512  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.796070 12536 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:17.728827 12671 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.796712 12671 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.797434  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.778221 12693 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:17.798048  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:17.798173 12693 consensus_queue.cc:1243] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:17.798391  8744 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:17.798841 12693 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.799288 12693 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:17.799712 12693 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.800402 12576 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:17.800840 12576 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:17.709781 12627 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.790236 12704 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.454s	user 0.003s	sys 0.000s
I20260709 13:01:17.801625 12576 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.803238 12576 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:17.804404  8789 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.804s (SENT)
I20260709 13:01:17.805624 12667 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.805984  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.794s (SENT)
I20260709 13:01:17.777204 12654 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:17.806079 12667 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.806543  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:17.639757 12540 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.807319 12643 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:17.807991  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.808239 12667 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:17.808894  8789 connection.cc:256] RPC call timeout handler was delayed by -1.4609634669999423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 13:01:17.809150 12540 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:17.796309  8824 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:17.796489  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.811995 12539 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.812580 12539 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.688522 12553 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:17.801550 12672 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:17.779417 12688 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.807957 12643 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:17.815668  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.792968  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.817265  8823 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:01:17.817477  8789 connection.cc:256] RPC call timeout handler was delayed by -1.4519634669999846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:01:17.839135  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.820633  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:17.781029 12706 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:17.796159  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1678) took 1100 ms. Trace:
I20260709 13:01:17.835760  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:17.848912  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2098100169999384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:01:17.837208  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:17.849033  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 56172.2us
W20260709 13:01:17.850183  8789 connection.cc:256] RPC call timeout handler was delayed by -1.1990100169999778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 13:01:17.850276  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.848991  8749 rpcz_store.cc:276] 0709 13:01:16.695749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.695858 (+   109us) service_pool.cc:224] Handling call
0709 13:01:17.795986 (+1100128us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540013b0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:17.796105 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249376512}
I20260709 13:01:17.768833  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:17.894490  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.851255  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:17.909339  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.851984  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.888967  8822 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:17.891038  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64681.7us
I20260709 13:01:17.910825  8743 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:17.849784  8741 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
W20260709 13:01:17.912352  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50674.7us
I20260709 13:01:17.732748 12682 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.913034  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:17.862057  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:17.862149  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.913838  8747 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.913988  8744 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.914170  8742 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:17.912086  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:17.835602 12641 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.001s
I20260709 13:01:17.970557  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.920673  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:17.940843  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.918234  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:17.919607  8816 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:17.822531 12713 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:17.974438  8824 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:17.832715 12580 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.975446  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 206727us
I20260709 13:01:17.984341  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.814872 12672 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:17.835279 12688 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.916153  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:17.979773  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59208.6us
W20260709 13:01:17.982686  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 131824us
I20260709 13:01:17.991458  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:17.913156 12626 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.889417 12486 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.985642  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.891283 12658 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.988386  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:17.861673 12531 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.996901  8747 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:17.989311  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:17.994110  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:17.995357 12597 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:17.996217 12198 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:17.978830 12690 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.979840 12571 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:17.801582 12627 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:17.999271  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:18.001475  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:18.003343  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.000797  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.010648  8744 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:18.012032  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:17.985478 12237 raft_consensus.cc:2764] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:17.982897 12654 raft_consensus.cc:1064] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:01:18.013756  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:18.032294  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:17.981559 12655 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.034911 12655 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.035022 12597 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:18.035498 12655 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.035584 12597 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.036108 12597 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.969715 12491 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:18.011633 12685 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:18.037653  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 607) took 1201 ms. Trace:
I20260709 13:01:18.038057  8897 rpcz_store.cc:276] 0709 13:01:16.836174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.848043 (+ 11869us) service_pool.cc:224] Handling call
0709 13:01:16.857507 (+  9464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:17.765000 (+907493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.037378 (+272378us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b540045abd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:18.037584 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055591296}
I20260709 13:01:17.974512 12666 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:01:17.977629 12641 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:18.039809  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 793) took 1509 ms (client timeout 1992 ms). Trace:
W20260709 13:01:18.040225  9447 rpcz_store.cc:269] 0709 13:01:16.530694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.532970 (+  2276us) service_pool.cc:224] Handling call
0709 13:01:18.039630 (+1506660us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:18.039749 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580231424}
I20260709 13:01:18.040827 12641 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.002765 12531 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.024121 12536 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:18.041566 12531 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.041779  9448 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:18.043682  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2000) took 1419 ms (client timeout 1564 ms). Trace:
W20260709 13:01:18.044037  8890 rpcz_store.cc:269] 0709 13:01:16.624085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.624244 (+   159us) service_pool.cc:224] Handling call
0709 13:01:18.043503 (+1419259us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:18.043629 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266231552}
I20260709 13:01:18.044832 12627 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.034957 12651 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.046434  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:17.997783 12713 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:18.047551 12651 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.047791  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.048662 12674 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:18.049078 12674 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:18.049499 12532 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:18.050165 12532 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:18.034139 12696 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:18.034391 12539 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.049952 12674 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.032976  8744 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:17.920368 12433 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
W20260709 13:01:18.052477  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 136442us
I20260709 13:01:18.055382  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.056490  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.004074 12658 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.051368 12634 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:18.064538 12634 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.064954 12634 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.020171 12233 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.998170 12701 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 13:01:18.089241  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 95151.6us
I20260709 13:01:18.013149 12626 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.057848 12680 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:18.053254 12696 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:18.058223 12536 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:18.071039 12658 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.002292 12580 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.080538 12674 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:18.108522  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59477.3us
I20260709 13:01:18.057924 12532 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:18.071973 12576 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:01:18.119836 12666 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.120154  9443 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:18.157644  9443 pending_rounds.cc:85] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 8
I20260709 13:01:18.157908  9443 pending_rounds.cc:107] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:18.120998 12542 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.126856  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:18.127178 12665 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.158135  8821 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:18.158640  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:18.127092  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:18.127907  9444 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:18.127993 12685 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.129015  9449 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:18.128793 12687 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:18.130053 12706 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.169133  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:18.170519  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:18.171204  9446 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:18.172173  9446 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:18.165891 12542 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.175151 12542 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.175767  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.176568  8743 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 7: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:18.110047 12237 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 13:01:18.190167  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.131102 12682 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:18.190712  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.190968 12682 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.191639 12682 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.134567 12233 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:18.192961 12233 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.132570  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.193674  8890 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:18.193430 12233 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.193997  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.194671  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.195261  8890 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:18.195330  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.196339 12627 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.136482  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.196794 12627 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.137857  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.097633 12648 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.198017 12633 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:18.198241  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 755) took 1027 ms. Trace:
I20260709 13:01:18.198565 12633 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:18.198608  9445 rpcz_store.cc:276] 0709 13:01:17.170885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.171012 (+   127us) service_pool.cc:224] Handling call
0709 13:01:18.198089 (+1027077us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:18.198188 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627755776}
I20260709 13:01:18.199448 12648 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.141402 12654 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.199576 12633 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.166970 12666 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.138367 12626 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.202631 12666 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.203117  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.057575 12571 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.135178  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.206055  8897 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:18.148970  8895 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:18.144742  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:18.150185  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.151043  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.207206  8819 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.144979 12580 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.211042  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:18.197216 12627 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.213914 12571 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.215106  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:18.215756  8815 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:18.208777  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:18.158152  9443 raft_consensus.cc:2889] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:18.160988  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:18.189734 12536 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.192174  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.156347 12680 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:18.221153  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.221791  8742 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:18.195509 12233 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.196218  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:18.203287  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.130672  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.253053  8740 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:01:18.253706  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1679) took 1557 ms (client timeout 1961 ms). Trace:
W20260709 13:01:18.254205  8740 rpcz_store.cc:269] 0709 13:01:16.696143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.752279 (+ 56136us) service_pool.cc:224] Handling call
0709 13:01:18.130525 (+1378246us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:18.253625 (+123100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72201600}
I20260709 13:01:18.151669  8898 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:18.149900  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.141834  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.142567 12198 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:18.210106  8744 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:18.210493 12679 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:18.107745 12671 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.218276  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.271236  8889 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.209722  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.271473  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.272173  8743 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:18.272125  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:18.275607  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:18.276482  8820 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.217192  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:18.219131  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:18.108537 12716 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:18.222227  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 174520us
I20260709 13:01:18.280619  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:18.222646  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.257362  8741 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:18.261183  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:18.262161  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 270800us
I20260709 13:01:18.283178  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.258535  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.216611  9444 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.287147  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:18.287812  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:18.278674  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.289350  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.289568  8740 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:18.290942  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.292799  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.293673  8740 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:18.296972  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:18.221107 12680 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:18.298791  8896 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:18.299058  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:18.303385  8742 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.249497 12580 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:18.261015 12616 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.302048  8746 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:18.300380  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.310882  8898 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:18.311149  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 219212us
I20260709 13:01:18.207789 12685 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:18.308317  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.309558  8741 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:18.267475 12679 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.262883 12198 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:18.313602  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.311990  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.318120  8891 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:18.326040  9444 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:18.332242  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.333092  8891 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:18.339558  8889 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:18.308009 12680 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:18.340939  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60433.2us
I20260709 13:01:18.307471 12539 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:18.344928  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.344569  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:18.267310 12693 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.219362 12696 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.279420 12655 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.306677 12724 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.319626 12314 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.318497 12679 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.358807  8742 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:18.359419  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:18.360129  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.407795  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.315157 12580 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.410933  8747 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:18.411141  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66291.5us
W20260709 13:01:18.411628  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 100573us
I20260709 13:01:18.354446 12716 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:18.320724 12616 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.420543  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:18.421303  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.421723  8889 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:18.350826 12723 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:18.383715 12690 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:18.381474 12724 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.353188 12667 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:01:18.414239 12540 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:18.355237 12626 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:01:18.464258  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 43942.6us
I20260709 13:01:18.407379 12571 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:01:18.467271  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 113012us
I20260709 13:01:18.441655 12728 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:18.499128  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77952.3us
I20260709 13:01:18.452481 12701 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.502177  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.501032  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:18.452775 12724 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:18.509995  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 165532us
I20260709 13:01:18.517709  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.518199  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.426484 12597 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.442103 12688 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.454089 12553 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:01:18.507079 12701 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.510052 12732 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:18.538312  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:18.539535  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.525417 12622 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.504647 12540 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:18.545563  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43871us
I20260709 13:01:18.516729 12682 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.443435 12198 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.559490 12658 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.561640 12751 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:18.547372 12731 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:18.533133 12486 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.563273 12486 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.565642 12655 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.566047 12690 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:18.550496  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:18.576395  8898 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:18.578025  8819 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:18.534762 12539 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:18.510011 12723 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:01:18.543810 12701 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.543958 12671 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:18.584365  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 83402.6us
I20260709 13:01:18.561916 12658 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.525321 12536 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:01:18.577807 12690 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.519306 12531 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.620057  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:18.584142 12739 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:01:18.583202 12622 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.623262 12597 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.610872 12698 raft_consensus.cc:2764] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:18.624451 12580 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.584923 12682 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:18.623957 12744 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:01:18.624933 12540 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:18.628145 12634 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.001s
I20260709 13:01:18.629197  8741 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:18.600454  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50089.4us
I20260709 13:01:18.567945 12687 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:18.634582  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.728s (SENT) [suppressed 56 similar messages]
W20260709 13:01:18.815096  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 180565us
I20260709 13:01:18.580538 12433 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:18.815651  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 195747us
I20260709 13:01:18.767096  8748 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:18.743539  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 225946us
W20260709 13:01:18.757428  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 179703us
I20260709 13:01:18.581378 12736 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:18.738214 12627 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:01:18.816965  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.834976  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:18.823472  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 277091us
I20260709 13:01:18.837210  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:18.732089 12724 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.823776 12634 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:18.840224 12747 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:01:18.852854  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.843114  8745 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:18.582098 12750 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.631599 12685 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.858595 12553 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.785007 12746 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.631099 12693 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:18.814491 12687 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.788826 12655 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:18.868319 12680 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.859350 12761 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.842998 12536 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:01:18.706362 12616 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.844590  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:18.823369 12723 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:18.630698 12688 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:18.836647 12654 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:18.864374  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50590.7us
I20260709 13:01:18.781034 12759 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:01:18.867807  9448 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:18.731405 12539 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:18.868185 12553 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:18.820348 12682 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.896754  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 603) took 1109 ms. Trace:
W20260709 13:01:18.897583  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44883.1us
I20260709 13:01:18.904600  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:18.900993  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:18.903172  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.902187  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:18.900349  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.913722  8892 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:18.847399 12627 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.862317 12750 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.863838 12758 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
W20260709 13:01:18.886865 12674 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.776s	user 0.006s	sys 0.005s
W20260709 13:01:19.138257  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 293821us
I20260709 13:01:18.978976  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.139195  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:19.142313  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:19.142728  8746 rpcz_store.cc:276] 0709 13:01:17.786923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.787086 (+   163us) service_pool.cc:224] Handling call
0709 13:01:18.896552 (+1109466us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400465ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:18.896695 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181232384}
I20260709 13:01:19.143298  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.143946  8890 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
W20260709 13:01:19.140589  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.149550  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 293194us
W20260709 13:01:19.056062 12728 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 13:01:19.166787  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.170205  8892 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:19.150490  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:19.171157  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:19.167073  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.167614  8891 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260709 13:01:19.140436 12665 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.925s	user 0.007s	sys 0.005s
I20260709 13:01:18.889070 12680 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:19.026379  8636 maintenance_manager.cc:419] P 5b6e9771cda34fa68ec7172c32ecd31b: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033337
I20260709 13:01:18.977466 12616 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.173671  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:18.955470 12667 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:19.026778 12753 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:01:19.141852 12654 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:18.881960 12687 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.895146 12634 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:19.140086 12682 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.138361 12553 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.977497 12442 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.901508 12757 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.140877 12698 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.977327 12762 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:19.141426 12536 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:19.233652 12536 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.142277 12750 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.238864 12723 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.239531 12723 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.170523 12699 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:19.241112 12699 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:19.241701 12699 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.914167 12746 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.868389 12754 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.307s
I20260709 13:01:18.915594 12597 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.259s
I20260709 13:01:19.172485  8890 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:19.253466  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 110239us
W20260709 13:01:19.254232  8787 leader_election.cc:336] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.564s (SENT)
W20260709 13:01:19.225081 12701 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.004s
W20260709 13:01:19.194798  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52574.9us
W20260709 13:01:19.196605  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 354020us
W20260709 13:01:19.178169 12760 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:01:18.881839 12655 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:18.877259 12486 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.175324 12728 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:19.178261 12531 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:19.174578 12688 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:19.184053 12616 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.225688 12622 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:19.229149 12729 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.229379 12442 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.231047 12769 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:19.254570  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.254616  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:19.193931 12778 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.185606 12665 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:19.266929  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:19.331008  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64206.3us
I20260709 13:01:19.210958 12757 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.332095 12757 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.331840 12690 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.232878 12762 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.275319  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:19.248286 12724 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:19.264770 12314 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.858s	user 0.010s	sys 0.002s
I20260709 13:01:19.297063  8890 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:19.261934 12667 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:19.302305 12442 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:19.302995 12486 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:19.302906 12764 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:01:19.301119 12643 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:19.304834  8898 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:19.305383 12536 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:19.315639 12746 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:19.265735 12759 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.323743 12679 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.002s
I20260709 13:01:19.331856  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:19.373593  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 191058us
I20260709 13:01:19.375054 12699 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.388321  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:19.404748  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 150565us
I20260709 13:01:19.410738  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.350580 12667 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:19.264446 12701 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.298205 12688 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.300324 12729 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.388109 12654 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.466688  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:19.491269  9415 leader_election.cc:336] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.514s (SENT)
I20260709 13:01:19.729056  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:19.508596  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 172929us
W20260709 13:01:19.730023  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 238845us
W20260709 13:01:19.467775  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 136021us
I20260709 13:01:19.389392 12556 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.734452  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.736419  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:19.737919  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:19.738698  8817 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.466513  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:19.759064  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:19.393158 12643 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:19.764290  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.767146  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.841732  9415 leader_election.cc:336] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.507s (SENT)
I20260709 13:01:19.794963  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.843345  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.402415 12754 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.397840 12486 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:19.846408  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:19.772183  8895 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:19.857024  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:19.645358 12761 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.003s
I20260709 13:01:19.851194  8889 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:19.852684  8892 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:19.466580 12542 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.669977 12680 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:19.847801  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 572614us
I20260709 13:01:19.849017  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.869935  8821 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:19.413453 12536 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:19.491807 12531 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:19.483081 12539 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:19.596791 12775 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.865005  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:19.881090  8892 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.866044  8816 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:19.691705 12690 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:19.506033 12723 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:19.872995  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.474443 12665 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:19.732234 12622 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:19.860967 12687 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.610s	user 0.005s	sys 0.005s
I20260709 13:01:19.855033 12654 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.871593 12761 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:19.884887  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.445391 12679 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:19.889837  9415 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.586s (SENT)
I20260709 13:01:19.933972  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:19.892815 12627 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:19.892376  8817 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:19.861888 12655 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:19.858860 12754 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.898666  8822 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:19.899330  8824 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:19.901834  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.867486 12680 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:19.851506 12693 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.880064 12531 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.843286 12556 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.944832  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.898643 12442 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.412499 12759 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.951233  8892 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:19.900609 12486 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:19.871965 12542 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.961514  8817 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:19.964097  8742 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:19.961216 12786 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:01:19.972088  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82324.1us
I20260709 13:01:19.973014  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:19.988010  8749 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:19.988320  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.990226  8824 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:19.988407  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:19.882091 12667 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.963033 12531 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.923233 12687 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:19.991303 12776 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.464s	user 0.001s	sys 0.000s
I20260709 13:01:19.918349 12643 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:20.019832  8712 leader_election.cc:336] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.527s (SENT)
W20260709 13:01:19.993666  9415 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 2.064s (SENT)
I20260709 13:01:20.021456  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.027953  8817 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.027475  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:19.966933 12622 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:19.844581 12781 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:01:19.963356 12794 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:19.899655 12775 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.974249 12761 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:01:20.028859  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:20.022440 12779 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 13:01:20.035184  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:19.965062 12693 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:19.968024 12442 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:19.968884 12672 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.046545 12795 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.058274  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2019) took 1158 ms. Trace:
I20260709 13:01:20.058677 12795 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.058801  8897 rpcz_store.cc:276] 0709 13:01:18.899954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.901572 (+  1618us) service_pool.cc:224] Handling call
0709 13:01:18.901882 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.902217 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.902259 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.902679 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.252782 (+350103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.252822 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.252889 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.058073 (+805184us) raft_consensus.cc:1685] finished
0709 13:01:20.058093 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.058216 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1554,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4220,"thread_start_us":213501,"threads_started":1,"wal-append.queue_time_us":487480}
W20260709 13:01:20.061944 12750 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.001s
I20260709 13:01:20.062407 12750 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:20.063805 12682 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.814s	user 0.010s	sys 0.000s
I20260709 13:01:20.064208 12682 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:20.065169 12778 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.736s	user 0.007s	sys 0.003s
I20260709 13:01:20.065596 12778 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:19.996073 12665 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:20.068128 12553 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.842s	user 0.002s	sys 0.009s
I20260709 13:01:20.068572 12553 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.060951 12795 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.048739  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.025746 12667 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.054630  8817 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:20.081740  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1985) took 2164 ms (client timeout 1892 ms). Trace:
I20260709 13:01:20.053483  8890 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:20.054865  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1694) took 2158 ms (client timeout 1773 ms). Trace:
I20260709 13:01:20.073349  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:20.052688  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59373.5us
W20260709 13:01:20.082360  8744 rpcz_store.cc:269] 0709 13:01:17.917255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.056287 (+139032us) service_pool.cc:224] Handling call
0709 13:01:20.081522 (+2025235us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540013c4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.081673 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350423424}
W20260709 13:01:20.236007  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 216254us
I20260709 13:01:20.224145  8821 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:20.082691  8747 rpcz_store.cc:269] 0709 13:01:17.896923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.000549 (+103626us) service_pool.cc:224] Handling call
0709 13:01:20.054667 (+2054118us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400463cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.054805 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871442688}
I20260709 13:01:20.056429 12622 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.049768 12781 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:20.258929  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.101020  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:20.295634  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.334426  8815 raft_consensus.cc:2376] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8. Re-sending same reply.
W20260709 13:01:20.335047  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.335315  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.335830  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.336251  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.198333  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:20.286266  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 440016us
W20260709 13:01:20.311183  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 324058us
W20260709 13:01:20.153906 12757 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.000s
I20260709 13:01:20.149207 12750 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:20.234999 12769 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.017s
W20260709 13:01:20.352994 12769 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.008s	sys 0.027s
I20260709 13:01:20.353392 12769 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:20.353981 12807 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.354331 12769 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.354676 12807 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.054366 12688 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.312692 12665 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.357745 12665 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:20.359565  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.365952  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.374277 12805 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.374956 12805 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.375391 12805 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.380923  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.315315 12729 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.267119 12682 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.266132 12553 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.255028 12531 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.297432 12597 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:20.251209 12800 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.256954 12759 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:20.248461 12777 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:01:20.323069  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.247159 12679 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.053166 12767 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:20.081449 12701 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:20.247793 12742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.602s	user 0.012s	sys 0.030s
I20260709 13:01:20.055124 12722 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:20.056178 12693 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.427578 12597 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:20.336985  8715 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.017s	sys 0.041s
W20260709 13:01:20.435667  8715 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.017s	sys 0.041s
W20260709 13:01:20.432258 12742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.801s	user 0.012s	sys 0.035s
I20260709 13:01:20.057799 12672 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.047099 12680 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.351526 12757 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.454624  8894 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.314122 12539 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:20.455785  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2021) took 1555 ms. Trace:
I20260709 13:01:20.455803 12757 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:20.453145  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.457470 12680 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:20.457680  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.399415  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62295.7us
I20260709 13:01:20.459460 12633 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.459945 12633 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.461599 12724 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.456159  8894 rpcz_store.cc:276] 0709 13:01:18.900674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.902925 (+  2251us) service_pool.cc:224] Handling call
0709 13:01:18.950975 (+ 48050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.455009 (+1504034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.455570 (+   561us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400249854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:20.455727 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3458204544}
I20260709 13:01:20.462172 12724 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.462745  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.463327  8894 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:20.464141 12724 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.464162  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:20.464764  9412 connection.cc:256] RPC call timeout handler was delayed by -1.5260016950000477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
W20260709 13:01:20.401083  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2015) took 1502 ms (client timeout 1501 ms). Trace:
I20260709 13:01:20.465601  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:20.465523  8889 rpcz_store.cc:269] 0709 13:01:18.898782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.900761 (+  1979us) service_pool.cc:224] Handling call
0709 13:01:20.400784 (+1500023us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.401005 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635305728}
I20260709 13:01:20.466990  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.467064  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.467832  8889 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:20.468335  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 620) took 1004 ms. Trace:
I20260709 13:01:20.052030 12775 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.058313 12442 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.081321 12699 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.402346  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2014) took 1504 ms. Trace:
I20260709 13:01:20.467031 12633 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.469453  8898 rpcz_store.cc:276] 0709 13:01:18.898457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.898625 (+   168us) service_pool.cc:224] Handling call
0709 13:01:18.899043 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.390014 (+490971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.402071 (+1012057us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54000483d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:20.402288 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932881152}
I20260709 13:01:20.470672  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.471288  8898 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:20.472184  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.472852  8898 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:01:20.473620  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.473754  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.474886  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.475780  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2255) took 1129 ms. Trace:
I20260709 13:01:20.472343  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.380991  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.269155 12687 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.469228 12556 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.148212 12797 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:20.477741 12797 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.480103 12797 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:20.153690 12754 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.424757  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.481534  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.482556  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.483078  8815 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:20.483556 12314 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.483899  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.484122 12314 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.484655  8815 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:20.402709 12665 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.335762 12781 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:20.485611  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.248781  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.489203 12696 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.490173  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.491052  8817 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.337960  8895 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:20.497566  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.498078  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 616) took 1158 ms. Trace:
I20260709 13:01:20.498349  8818 raft_consensus.cc:2376] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:01:20.428653 12682 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:20.499300 12682 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.499060  8895 rpcz_store.cc:276] 0709 13:01:19.340030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.156584 (+816554us) service_pool.cc:224] Handling call
0709 13:01:20.498001 (+341417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.501574 12682 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:20.438789  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.502400  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.502959  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.503590  8824 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:20.504047  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.504199  8895 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:20.504370  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.504746  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2027) took 1407 ms. Trace:
I20260709 13:01:20.429800 12800 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.439790 12531 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.436968 12693 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.505951 12693 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.506994 12800 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:20.507604  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 802) took 2381 ms. Trace:
I20260709 13:01:20.508155 12693 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.433300 12722 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:20.508021  9448 rpcz_store.cc:276] 0709 13:01:18.126141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.128370 (+  2229us) service_pool.cc:224] Handling call
0709 13:01:18.205327 (+ 76957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.144632 (+939305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.506635 (+1362003us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:20.507383 (+   748us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54006801d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.507517 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699331584}
I20260709 13:01:20.509698  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.509495 12722 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.510332  8822 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:20.510823 12722 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.512903 12637 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.513868 12637 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:20.514703  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 264284us
I20260709 13:01:20.505218  8824 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:20.440732 12698 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:20.516546 12637 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.517204 12698 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:20.518065  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 607) took 2699 ms (client timeout 1728 ms). Trace:
W20260709 13:01:20.518487  8745 rpcz_store.cc:269] 0709 13:01:17.818795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.835464 (+ 16669us) service_pool.cc:224] Handling call
0709 13:01:20.517863 (+2682399us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54000b6654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.517995 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285270272}
I20260709 13:01:20.519563  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.520928  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.442078  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.442559 12679 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.443776 12597 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:20.445206  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2294) took 2466 ms (client timeout 1789 ms). Trace:
I20260709 13:01:20.544093 12679 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:20.546015  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.547219  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.547760  8824 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:20.446379  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2251) took 1100 ms. Trace:
I20260709 13:01:20.549443  8890 rpcz_store.cc:276] 0709 13:01:19.345777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.856713 (+510936us) service_pool.cc:224] Handling call
0709 13:01:20.446233 (+589520us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000aafd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.446323 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450187136}
I20260709 13:01:20.551244  8890 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:20.446647  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.551445  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.552801  8894 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:20.465119  9412 connection.cc:256] RPC call timeout handler was delayed by -1.5035016950000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 13:01:20.553447  9412 connection.cc:256] RPC call timeout handler was delayed by -1.4261016949999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 13:01:20.553791  9412 connection.cc:256] RPC call timeout handler was delayed by -1.4063016950000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 13:01:20.554092  9412 connection.cc:256] RPC call timeout handler was delayed by -1.3271016950000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:01:20.554431  9412 connection.cc:256] RPC call timeout handler was delayed by -1.2307494469999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:01:20.468861  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2022) took 1567 ms (client timeout 1564 ms). Trace:
I20260709 13:01:20.476085  8898 rpcz_store.cc:276] 0709 13:01:19.346735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.474663 (+1127928us) service_pool.cc:224] Handling call
0709 13:01:20.475728 (+  1065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.268276 12542 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:20.477149  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96277.1us
W20260709 13:01:20.559482  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.558811  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:20.543898  8823 rpcz_store.cc:269] 0709 13:01:17.979278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.980024 (+   746us) service_pool.cc:224] Handling call
0709 13:01:20.444991 (+2464967us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.445122 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136872192}
I20260709 13:01:20.505100  8895 rpcz_store.cc:276] 0709 13:01:19.097643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.503441 (+1405798us) service_pool.cc:224] Handling call
0709 13:01:20.503941 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.504538 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.504700 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.486786 12314 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.486498 12781 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:20.486903  8715 connection.cc:256] RPC call timeout handler was delayed by -1.0213531269999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 13:01:20.487445 12699 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.050016 12794 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.460215  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.488090  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.488474  8897 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:20.575531  8815 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:20.468683  8889 rpcz_store.cc:276] 0709 13:01:19.463964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.466740 (+1002776us) service_pool.cc:224] Handling call
0709 13:01:20.468279 (+  1539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20608}
I20260709 13:01:20.498661  8896 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 7.
I20260709 13:01:20.430401 12807 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.441454  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.625777  8889 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:20.629957  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.630434  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.631353  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 609) took 1370 ms. Trace:
I20260709 13:01:20.631702  8889 rpcz_store.cc:276] 0709 13:01:19.260592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.576760 (+1316168us) service_pool.cc:224] Handling call
0709 13:01:20.577117 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.630026 (+ 52909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.631297 (+  1271us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:20.523264  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.536437  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.632901  8889 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:20.633394  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:20.634305  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.635316  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.636298  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.537706  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.637630  8822 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:20.542599  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:20.543239  8745 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:20.556115  8891 rpcz_store.cc:269] 0709 13:01:18.900947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.904395 (+  3448us) service_pool.cc:224] Handling call
0709 13:01:20.468677 (+1564282us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:20.468797 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58623232}
I20260709 13:01:20.630010  8747 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.644289  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 610) took 1383 ms. Trace:
I20260709 13:01:20.644932  8891 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:20.644704  8889 rpcz_store.cc:276] 0709 13:01:19.260947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.632436 (+1371489us) service_pool.cc:224] Handling call
0709 13:01:20.632737 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.643846 (+ 11109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.644228 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.645768  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 612) took 1381 ms. Trace:
I20260709 13:01:20.646095  8891 rpcz_store.cc:276] 0709 13:01:19.264601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.640380 (+1375779us) service_pool.cc:224] Handling call
0709 13:01:20.640689 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.645532 (+  4843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.645721 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:20.632984  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.649402  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.560482  8824 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:20.563839  8817 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:20.558964  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.566306  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.654150  8748 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:20.654386  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.570333  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2029) took 1384 ms. Trace:
I20260709 13:01:20.573868  8898 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.655086  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.655900  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2032) took 1425 ms. Trace:
W20260709 13:01:20.610059  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 149923us
I20260709 13:01:20.549960 12698 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.557858 12571 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:20.592902  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.609200  8895 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:20.628715  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2030) took 1398 ms. Trace:
I20260709 13:01:20.661717  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2033) took 1430 ms. Trace:
I20260709 13:01:20.662492  8895 rpcz_store.cc:276] 0709 13:01:19.250030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.560841 (+1310811us) service_pool.cc:224] Handling call
0709 13:01:20.608848 (+ 48007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.661368 (+ 52520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.661645 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:20.663105  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104275us
I20260709 13:01:20.663738  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.656353  8898 rpcz_store.cc:276] 0709 13:01:19.230764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.556486 (+1325722us) service_pool.cc:224] Handling call
0709 13:01:20.573640 (+ 17154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.655522 (+ 81882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.655710 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.664304  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.610816  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:20.665222  8823 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:20.654954  8890 rpcz_store.cc:276] 0709 13:01:19.174596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.550455 (+1375859us) service_pool.cc:224] Handling call
0709 13:01:20.550913 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.558141 (+  7228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.558430 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.657994  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.656002  8819 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.659586  8889 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:20.659891  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.576190 12679 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.575066 12699 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:20.669027  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:20.669205  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.664033  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.666615  8815 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:20.670420  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 614) took 1364 ms. Trace:
I20260709 13:01:20.670985  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.670918  8891 rpcz_store.cc:276] 0709 13:01:19.283842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.646994 (+1363152us) service_pool.cc:224] Handling call
0709 13:01:20.647260 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.647433 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.647457 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.647508 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:20.647540 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.647591 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.647681 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.672359  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:20.667873  8817 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:20.672428  8824 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:20.675424  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.675450  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.666671  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2034) took 1396 ms. Trace:
I20260709 13:01:20.677208  8889 rpcz_store.cc:276] 0709 13:01:19.269916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.645860 (+1375944us) service_pool.cc:224] Handling call
0709 13:01:20.659292 (+ 13432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.666395 (+  7103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.666615 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.676239  8746 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.674001  8787 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.683708  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.684542  8748 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:20.658919  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.692076  8898 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:20.692739  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2254) took 1346 ms. Trace:
I20260709 13:01:20.693101  8898 rpcz_store.cc:276] 0709 13:01:19.346493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.664703 (+1318210us) service_pool.cc:224] Handling call
0709 13:01:20.691820 (+ 27117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.692489 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.692686 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.696321  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.626811 12314 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.661576 12637 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.705109  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.708729  8823 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:20.706147  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.709501  8747 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.709328  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.710166  8815 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.710574  8889 raft_consensus.cc:2410] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:20.704315  8745 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:20.719379  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.719973  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.669113 12353 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.724239  8818 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:20.723726  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.673359 12807 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.729259  8824 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.729005  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.725944  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:20.715410  8787 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.974s (SENT)
I20260709 13:01:20.746984  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.723937  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.686678 12699 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:20.674181 12724 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:20.673138 12693 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.673732 12597 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.672930 12759 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.763026  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.681761 12542 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.738127  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.740945  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.691402 12553 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.749784  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.729861 12803 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.752228  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.756810  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.780314  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.757174  8819 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:20.703202 12680 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:20.705456 12679 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.761620  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.707870 12672 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:01:20.762450 12800 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.771133  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.682080 12769 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:01:20.772122  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.728966 12665 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:20.773301  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:20.795085  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:20.795347  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86938.4us
I20260709 13:01:20.795578  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.765967 12759 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.796680  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.794898  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:20.691602 12531 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:20.801570  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.776446 12755 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.674996 12571 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.737026 12353 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.783151  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.784488  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.743007 12442 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.766523  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.808578  8745 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.787909  8746 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.758175 12729 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:20.762661 12783 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.758821 12314 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.764878  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.768990 12542 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.794588  8748 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.774535 12637 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:20.798026  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.790792  8747 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:20.802616  8898 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:20.733187 12233 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.773084 12553 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:20.783376 12767 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.786043 12693 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:20.806893  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 137773us
I20260709 13:01:20.847994  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:20.848779 12633 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.785995 12679 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:20.849531 12679 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.849996 12571 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:20.850509 12553 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.771783 12597 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.853117 12597 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:20.832793 12822 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.799674 12759 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:20.854038  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.855300 12812 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:01:20.823652  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.801568 12682 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.857113 12794 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.830004  8745 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:20.857496 12682 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.857990 12682 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.858369  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.859583  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.806015 12755 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.860242 12755 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:20.821261  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56553us
I20260709 13:01:20.861198  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:20.861838 12818 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:01:20.863210 12819 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:01:20.864283 12813 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:20.807229 12353 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.805527 12665 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:20.865993  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.866159 12665 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.866818  8744 raft_consensus.cc:2410] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:01:20.867616  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.838515  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.788156 12680 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.812057 12531 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:20.833799  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:20.801237 12732 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:20.820211 12729 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.833091 12805 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.927502  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.929749  8746 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.929896  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.831067 12637 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:20.925395  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.830777 12542 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.819831 12806 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:20.837903 12233 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.945720  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.958396  8746 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.928704 12732 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.959151 12732 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.849157  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:20.960953  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111873us
I20260709 13:01:20.822916 12783 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.967595 12805 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.971833 12682 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.974363 12722 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.974891 12722 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.975265 12722 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.976128  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 618) took 1579 ms. Trace:
I20260709 13:01:20.954161 12597 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.976974 12597 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:20.977250  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.977640 12597 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.978006  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:20.977684 12531 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.978966  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.980131  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:20.980556 12755 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:20.946859 12637 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:20.976545  8890 rpcz_store.cc:276] 0709 13:01:19.396798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.674211 (+1277413us) service_pool.cc:224] Handling call
0709 13:01:20.674560 (+   349us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:20.684636 (+ 10076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.684682 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:20.685617 (+   935us) log.cc:844] Serialized 78 byte log entry
0709 13:01:20.831709 (+146092us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:20.928230 (+ 96521us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.957565 (+ 29335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.957652 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.975876 (+ 18224us) raft_consensus.cc:1685] finished
0709 13:01:20.975897 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.976054 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38575,"prepare.queue_time_us":96299,"raft.queue_time_us":44601,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":12881,"thread_start_us":55942,"threads_started":1,"wal-append.queue_time_us":219276}
I20260709 13:01:20.982349  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.983050  8890 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:20.984725  8743 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:20.984169  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.880653  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.842839  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.987553  8898 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:20.943766  8818 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:20.927626  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 93916.2us
I20260709 13:01:20.988694  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:20.815632 12442 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.869753  8747 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:20.989650  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:20.879844 12822 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.842525 12800 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:20.991218  8890 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:20.990001  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 164525us
I20260709 13:01:20.871482  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 610) took 3048 ms. Trace:
I20260709 13:01:21.020706  8740 rpcz_store.cc:276] 0709 13:01:17.823159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.294745 (+471586us) service_pool.cc:224] Handling call
0709 13:01:18.295365 (+   620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.345585 (+ 50220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.345615 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.350595 (+  4980us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.523981 (+173386us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:18.524019 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.838832 (+2314813us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54001d9554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:20.838890 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.838964 (+    74us) raft_consensus.cc:1685] finished
0709 13:01:20.838983 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.871412 (+ 32429us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":681,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11345,"spinlock_wait_cycles":114620288,"thread_start_us":117388,"threads_started":1,"wal-append.queue_time_us":193710,"wal-append.run_cpu_time_us":1627,"wal-append.run_wall_time_us":1001646}
I20260709 13:01:21.022890  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:21.001330  8744 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:20.831198 12314 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.023859  8740 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:21.024885  8740 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:21.026203  8740 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 13:01:21.027016 12442 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.002031 12698 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.002570 12353 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:20.992956  8898 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:21.003139  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:20.989768  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:21.027479  8740 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.032408  8818 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:21.035449  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:21.037735  8748 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:21.001896 12745 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:01:21.009079  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:21.009423 12633 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:21.010200 12811 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:01:21.014240  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:20.985952 12233 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.957312 12806 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:21.022946  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:20.995133  8746 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:21.084688  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:21.370258  8890 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:01:21.361990  8862 leader_election.cc:336] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.793s (SENT)
W20260709 13:01:21.048058  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.376227  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:21.336081  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.378314  8740 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.147436  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1691) took 3251 ms. Trace:
W20260709 13:01:21.105865  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.150584  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2035) took 1422 ms. Trace:
I20260709 13:01:21.379392  8742 rpcz_store.cc:276] 0709 13:01:17.895923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.406521 (+510598us) service_pool.cc:224] Handling call
0709 13:01:18.425623 (+ 19102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.971005 (+1545382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.147099 (+1176094us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400462dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:21.147355 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3538308992}
I20260709 13:01:21.335291  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.380020  8897 rpcz_store.cc:276] 0709 13:01:19.728092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.340547 (+612455us) service_pool.cc:224] Handling call
0709 13:01:21.150328 (+809781us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400372154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.150500 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279710464}
I20260709 13:01:21.098726  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:21.374496  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 384966us
I20260709 13:01:21.384451  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:21.369647  8747 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
W20260709 13:01:21.269959  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1696) took 3358 ms (client timeout 1529 ms). Trace:
I20260709 13:01:21.390127  8742 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:01:21.390583  8742 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 4
I20260709 13:01:21.370668  8898 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:21.390894  8742 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:01:21.391306  8742 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:21.390693  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:21.391748  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:21.392321  8897 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.392719  8898 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:01:21.389986  8749 rpcz_store.cc:269] 0709 13:01:17.911615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.969636 (+ 58021us) service_pool.cc:224] Handling call
0709 13:01:21.269667 (+3300031us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b540040c9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.269856 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4576558464}
I20260709 13:01:21.133157  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:21.061972  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 71674.5us
W20260709 13:01:21.098481  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 439703us
I20260709 13:01:21.398622  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:21.021528 12800 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.401727 12800 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.406033 12682 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.138497 12699 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:21.412658  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.413185  8742 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.394073  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:21.411895  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:21.413232  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.416211  8819 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
W20260709 13:01:21.416455  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.417155  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:21.418761  8742 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:21.419127  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.421612  8747 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:21.421777  8749 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:21.323349 12724 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.009s
W20260709 13:01:21.420531  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 58621.2us
I20260709 13:01:21.423576  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:21.424167  8819 raft_consensus.cc:2376] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:01:21.424258  8742 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:21.425000  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.425853  8819 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:21.426550  8890 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:21.426729  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.427278  8819 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:21.428097  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.428764  8819 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:21.433010  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.433904  8818 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:21.025161 12314 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:21.414314 12759 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 13:01:21.393345 12633 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.363198 12353 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.438426  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.130853 12680 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.146606 12755 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:21.082826 12622 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:21.443033  8749 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.136057 12637 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.448750  8897 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:21.389490 12806 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:21.435271 12724 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.007s	sys 0.010s
I20260709 13:01:21.455154  8898 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:21.455313  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.196533 12542 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:21.444482  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.436684 12750 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.438272 12783 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.436457 12784 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:21.465917 12314 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:21.477416  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.446519 12811 consensus_queue.cc:1243] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:21.447203 12682 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:21.465451 12665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.010s
I20260709 13:01:21.481447  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.453415 12237 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.497212  8747 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:21.415167 12758 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:21.474687 12616 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.490996  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:21.491803 12665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.017s	sys 0.010s
I20260709 13:01:21.495208 12648 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:21.484939 12314 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.495704  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:21.511245 12597 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:01:21.482226 12822 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:21.527568  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 129133us
I20260709 13:01:21.528395  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:21.447611 12759 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.484625 12531 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:21.488559 12679 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.000s
I20260709 13:01:21.454514 12633 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.522296  8818 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:21.488377 12783 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:21.492261 12622 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.487398 12682 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:21.524840  8895 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:21.514134 12729 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:21.530730  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2013) took 2626 ms (client timeout 1974 ms). Trace:
I20260709 13:01:21.499099 12831 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:21.512192 12237 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.530110  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:21.516532 12648 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:21.545399  8896 rpcz_store.cc:269] 0709 13:01:18.898092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.915435 (+ 17343us) service_pool.cc:224] Handling call
0709 13:01:21.524078 (+2608643us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.524211 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123495808}
I20260709 13:01:21.487833 12784 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.519845 12314 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.523765 12580 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:21.561919  8824 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:21.568147 12791 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.625s	user 0.001s	sys 0.001s
I20260709 13:01:21.570410 12542 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.531747 12531 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:21.575894  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45876.1us
I20260709 13:01:21.535267 12783 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.551865 12822 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.582855  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:21.548187 12831 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:21.563482 12722 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.002s
I20260709 13:01:21.532850 12597 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.574558 12648 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.574331 12637 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.574376 12237 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.578317 12622 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.596625  9412 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.579990 12353 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.541810 12679 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:21.585249  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 201021us
I20260709 13:01:21.581082 12580 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:21.584120 12531 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.588145 12783 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.619335  8896 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.591042 12831 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.624258  8742 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.622831 12822 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:21.591809 12722 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.621594 12837 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:21.653352 12835 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:01:21.598456 12682 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.654898  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1991) took 1078 ms. Trace:
I20260709 13:01:21.655079 12682 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.655471  8746 rpcz_store.cc:276] 0709 13:01:20.576395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.375344 (+798949us) service_pool.cc:224] Handling call
0709 13:01:21.375695 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.440315 (+ 64620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.440355 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.440808 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.591038 (+150230us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.591086 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.591163 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.654582 (+ 63419us) raft_consensus.cc:1685] finished
0709 13:01:21.654615 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.654818 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64193,"raft.queue_time_us":67959,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":775,"thread_start_us":107038,"threads_started":1,"wal-append.queue_time_us":184363}
I20260709 13:01:21.670689 12679 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.676859 12759 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:21.678227 12833 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:01:21.685813 12837 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:21.686304 12837 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:21.688998 12732 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.002s
I20260709 13:01:21.722535 12732 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.723843 12633 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.649196 12822 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.800894 12822 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.694116 12784 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:21.599299 12622 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.807646 12622 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:21.694623 12553 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.844s	user 0.009s	sys 0.003s
I20260709 13:01:21.808326 12553 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.697620 12580 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.810698 12822 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.639321 12803 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:01:21.812114 12803 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:21.700450  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2020) took 2790 ms. Trace:
I20260709 13:01:21.702399 12722 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.702656 12198 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:21.812856  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:21.707542 12433 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:21.712159  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2001) took 1029 ms. Trace:
I20260709 13:01:21.718902  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 621) took 1857 ms. Trace:
I20260709 13:01:21.812662  8819 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.835949  8747 rpcz_store.cc:276] 0709 13:01:20.682599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.422386 (+739787us) service_pool.cc:224] Handling call
0709 13:01:21.446222 (+ 23836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.497655 (+ 51433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.711753 (+214098us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b54000217d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:21.712062 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116749184}
W20260709 13:01:21.819197  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.837323  8816 rpcz_store.cc:276] 0709 13:01:19.861642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.861795 (+   153us) service_pool.cc:224] Handling call
0709 13:01:19.862159 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.889722 (+ 27563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.718612 (+1828890us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b540029a4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:21.718835 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8198912}
I20260709 13:01:21.839068  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:21.719887 12806 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.721398 12800 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:21.862099  8644 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 269) took 40323 ms (client timeout 15000 ms). Trace:
I20260709 13:01:21.864125 12800 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.691655 12233 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.726594 12732 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.733076  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1703) took 1290 ms. Trace:
I20260709 13:01:21.733805  8749 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:21.865849  8748 rpcz_store.cc:276] 0709 13:01:20.442244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.834004 (+391760us) service_pool.cc:224] Handling call
0709 13:01:21.732853 (+898849us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b540013c9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.733017 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109100928}
I20260709 13:01:21.734937  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:21.866634 12233 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.866909  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:21.867530  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 132747us
I20260709 13:01:21.737452 12762 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:21.740710  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 622) took 1861 ms (client timeout 1618 ms). Trace:
I20260709 13:01:21.870081  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:21.742151  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1714) took 1297 ms. Trace:
I20260709 13:01:21.916077  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:21.916245  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46244.6us
I20260709 13:01:21.916366  8740 rpcz_store.cc:276] 0709 13:01:20.444979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.033397 (+588418us) service_pool.cc:224] Handling call
0709 13:01:21.378083 (+344686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.378927 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.741876 (+362949us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b54000b5c50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.742077 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8829824}
I20260709 13:01:21.917392  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:21.744706 12831 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.747984 12648 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.924320  8740 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.749521  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 1987) took 1178 ms. Trace:
I20260709 13:01:21.925263  8744 rpcz_store.cc:276] 0709 13:01:20.571069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.867388 (+296319us) service_pool.cc:224] Handling call
0709 13:01:21.749249 (+881861us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400464bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.749440 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306193152}
I20260709 13:01:21.927011  8744 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.927824  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2007) took 1118 ms. Trace:
I20260709 13:01:21.928243  8744 rpcz_store.cc:276] 0709 13:01:20.808994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.926383 (+1117389us) service_pool.cc:224] Handling call
0709 13:01:21.926781 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.927522 (+   741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.927744 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.750861 12811 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:21.755129 12643 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:21.756524  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2321) took 1008 ms. Trace:
I20260709 13:01:21.756963 12725 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:21.757874  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1679) took 1217 ms. Trace:
I20260709 13:01:21.758445 12752 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.759670  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:21.764240  8862 leader_election.cc:336] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.992s (SENT)
I20260709 13:01:21.956579  8817 rpcz_store.cc:276] 0709 13:01:20.540654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.691770 (+151116us) service_pool.cc:224] Handling call
0709 13:01:20.692125 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.692404 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:20.815981 (+123577us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.757517 (+941536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.757583 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:21.757631 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.757697 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.757816 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":868279,"prepare.queue_time_us":95223,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":824107,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":183103,"threads_started":2,"wal-append.queue_time_us":205293}
I20260709 13:01:21.960238  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:21.961227  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:21.961557  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 197436us
W20260709 13:01:21.962505  8862 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 2.002s (SENT)
I20260709 13:01:21.963017  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:21.767434  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:21.963824  8889 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.963938  8817 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:21.772706  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 1.02259e+06us
I20260709 13:01:21.965934  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:21.732806 12314 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.778136 12699 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:20.661433  8894 rpcz_store.cc:276] 0709 13:01:19.229894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.553459 (+1323565us) service_pool.cc:224] Handling call
0709 13:01:20.553733 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.553954 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:20.554220 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.627158 (+ 72938us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.627220 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:20.627276 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.627348 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.628609 (+  1261us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":50,"raft.queue_time_us":912,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":30311,"thread_start_us":72428,"threads_started":1}
I20260709 13:01:21.781015  8745 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:21.786151  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:21.923689 12556 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:21.787819  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.791292  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.792755 12539 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.793649  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:21.794133  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2250) took 2448 ms. Trace:
I20260709 13:01:21.794915 12654 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:21.797276 12783 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.799973 12531 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.698359 12597 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:21.817862  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.703878 12706 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.842032 12810 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.842720 12433 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:21.843612 12553 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.820935 12826 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.812476 12803 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:21.821182  8893 rpcz_store.cc:276] 0709 13:01:18.900373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.902795 (+  2422us) service_pool.cc:224] Handling call
0709 13:01:18.903057 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.189384 (+286327us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:19.415507 (+226123us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.969489 (+553982us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.992714 (+ 23225us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.992907 (+   193us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.992961 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.993062 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.628184 (+1635122us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b2400044bb8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f3420fb0 00007f90f6097488 00007f90f6097be8 00007f90f606f398 00007f90f606a76a 00007f90f6069bb6
0709 13:01:21.690042 (+ 61858us) raft_consensus.cc:1685] finished
0709 13:01:21.690189 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.690450 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1425,"prepare.queue_time_us":54944,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":125957,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":57813,"spinlock_wait_cycles":45167360,"thread_start_us":945414,"threads_started":4,"wal-append.queue_time_us":494653}
I20260709 13:01:21.853991 12633 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.857278 12722 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:21.739383 12781 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.934312 12831 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.936574 12648 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.921512 12807 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.924186 12665 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:21.921945 12837 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.931210 12729 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:21.774906 12768 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 2.581s	user 0.005s	sys 0.014s
I20260709 13:01:21.965745 12762 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.971500 12314 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.786898 12751 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.923483 12233 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:21.968104 12616 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:21.931488 12811 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:22.043314 12616 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.968683 12752 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.043983 12811 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.045662 12811 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.045792 12752 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:21.979636 12732 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.028301 12803 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.984159 12788 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:01:21.991703 12742 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:01:21.991974 12654 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:21.979502 12667 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:21.943378 12806 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:21.966385 12725 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.033088 12826 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.045433 12751 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.034379 12722 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.051247 12783 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.051227 12822 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.056860 12810 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:22.059531 12810 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.060897 12722 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:21.991523 12830 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:22.056059 12732 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:22.072065 12837 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.071214 12803 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.857895  8643 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 5, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 5 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:21.862514  8644 rpcz_store.cc:269] 0709 13:00:41.538974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.590790 (+ 51816us) service_pool.cc:224] Handling call
0709 13:01:21.862003 (+40271213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:13.313432 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:13.314241 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:13.314304 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:01:13.314324 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:13.315289 (+   965us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:13.315373 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:13.315400 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:01:13.316051 (+   651us) tablet.cc:701] Row locks acquired
0709 13:01:13.316111 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:01:13.316301 (+   190us) write_op.cc:270] Start()
0709 13:01:13.316462 (+   161us) write_op.cc:276] Timestamp: P: 1783602073316231 usec, L: 0
0709 13:01:13.316488 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:01:13.385520 (+ 69032us) log.cc:844] Serialized 2462 byte log entry
0709 13:01:14.155530 (+770010us) op_driver.cc:464] REPLICATION: finished
0709 13:01:14.410552 (+255022us) write_op.cc:301] APPLY: starting
0709 13:01:14.410701 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:14.410969 (+   268us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:14.411171 (+   202us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:14.482543 (+ 71372us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:14.677221 (+194678us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:01:14.677276 (+    55us) write_op.cc:312] APPLY: finished
0709 13:01:14.677618 (+   342us) log.cc:844] Serialized 53 byte log entry
0709 13:01:14.677819 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:14.677942 (+   123us) write_op.cc:454] Released schema lock
0709 13:01:14.678035 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:14.742865 (+ 64830us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:14.747015 (+  4150us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":809521,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":245839,"apply.run_cpu_time_us":4975,"apply.run_wall_time_us":286004,"num_ops":6,"prepare.queue_time_us":175303,"prepare.run_cpu_time_us":23767,"prepare.run_wall_time_us":342427,"raft.queue_time_us":390126,"raft.run_cpu_time_us":7534,"raft.run_wall_time_us":211982,"replication_time_us":818936,"thread_start_us":615774,"threads_started":4,"wal-append.queue_time_us":328565,"wal-append.run_cpu_time_us":40039,"wal-append.run_wall_time_us":2208401}]]}
I20260709 13:01:21.920002  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:22.121953  8643 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 5 to 6, leader changed from <none> to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 6 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:22.128592  8643 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 5 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:01:22.131232  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 211258us
I20260709 13:01:21.931788  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 633) took 1185 ms. Trace:
I20260709 13:01:22.135730  8740 rpcz_store.cc:276] 0709 13:01:20.746160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.923661 (+1177501us) service_pool.cc:224] Handling call
0709 13:01:21.924079 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.924914 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.931730 (+  6816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.139534  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:21.811065  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 625) took 1067 ms. Trace:
I20260709 13:01:22.141767  8742 rpcz_store.cc:276] 0709 13:01:20.743560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.425349 (+681789us) service_pool.cc:224] Handling call
0709 13:01:21.446371 (+ 21022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.640888 (+194517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.810753 (+169865us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400358c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:21.810991 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408849664}
I20260709 13:01:21.930187  8744 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.146809  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2008) took 1337 ms. Trace:
I20260709 13:01:22.147375  8744 rpcz_store.cc:276] 0709 13:01:20.809349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.929080 (+1119731us) service_pool.cc:224] Handling call
0709 13:01:21.930023 (+   943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.145877 (+215854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.146432 (+   555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.136579  8643 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.849686  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:21.867583  8748 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.865576  8815 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:21.917639  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:21.929522  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.136266  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:21.866715  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 627) took 1122 ms. Trace:
I20260709 13:01:22.134601  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 620) took 4078 ms. Trace:
I20260709 13:01:21.854578  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.156631  8816 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:21.997704  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 346957us
I20260709 13:01:21.845959  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2315) took 1194 ms. Trace:
I20260709 13:01:22.158123  8819 rpcz_store.cc:276] 0709 13:01:20.651882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.437971 (+786089us) service_pool.cc:224] Handling call
0709 13:01:21.438278 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.845116 (+406838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.845714 (+   598us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:21.845890 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861009664}
I20260709 13:01:22.158829  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:21.967569  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.159885  8816 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:22.160223  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.160539  8817 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:21.915608  8821 rpcz_store.cc:269] 0709 13:01:19.879651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.879960 (+   309us) service_pool.cc:224] Handling call
0709 13:01:21.740468 (+1860508us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.740619 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610504448}
I20260709 13:01:22.162379  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.163012  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:21.973519  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 187544us
I20260709 13:01:22.164791  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:22.167446  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.168026  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:22.168800  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.966377  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:22.170334  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.171414  9440 raft_consensus.cc:2376] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10. Re-sending same reply.
W20260709 13:01:22.171551  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 205206us
I20260709 13:01:21.979717  8889 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:21.975570 12556 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:22.173519  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2262) took 1467 ms (client timeout 1628 ms). Trace:
I20260709 13:01:22.173882 12556 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.174041  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.174850  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.175864  9447 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.154768  8749 rpcz_store.cc:276] 0709 13:01:20.744268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.466982 (+722714us) service_pool.cc:224] Handling call
0709 13:01:21.733447 (+266465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.865926 (+132479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.866534 (+   608us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540013c9d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.866657 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582569856}
I20260709 13:01:21.978356  8895 rpcz_store.cc:276] 0709 13:01:19.345480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.663306 (+1317826us) service_pool.cc:224] Handling call
0709 13:01:21.524516 (+861210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.542403 (+ 17887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.793942 (+251539us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54000497d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.794075 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908452608}
I20260709 13:01:22.177584 12233 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.178282  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:22.178862  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:22.179647  8749 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:22.179894 12233 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.052265 12531 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.166208  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.177614  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.185545 12531 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.990310 12539 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:22.153970  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46895.9us
I20260709 13:01:22.186532  8819 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.188715 12539 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.052657 12433 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:21.949645  8824 rpcz_store.cc:276] 0709 13:01:20.748152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.782671 (+ 34519us) service_pool.cc:224] Handling call
0709 13:01:21.756302 (+973631us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54000820d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:21.756460 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762336512}
I20260709 13:01:22.168998  8821 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.163230  8823 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:22.183290  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.161135  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:22.150768  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 164653us
I20260709 13:01:22.175145  9448 raft_consensus.cc:2368] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:22.144920  8742 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:22.140956  8740 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.210343  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:22.211199  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.210855 12732 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.212432 12682 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:22.213899  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:22.174008  8889 rpcz_store.cc:269] 0709 13:01:20.705838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.767141 (+1061303us) service_pool.cc:224] Handling call
0709 13:01:22.173140 (+405999us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.213107 12722 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:01:22.229951 12616 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.189684  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.208236  8643 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 5 to 8, leader changed from <none> to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129), now has a pending config: VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129), VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130), VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131), VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:22.245554  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 636) took 1233 ms. Trace:
I20260709 13:01:22.244879  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 635) took 1254 ms. Trace:
I20260709 13:01:22.247718  8742 rpcz_store.cc:276] 0709 13:01:20.987693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.143624 (+1155931us) service_pool.cc:224] Handling call
0709 13:01:22.144426 (+   802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.241799 (+ 97373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.242083 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.246615  8744 rpcz_store.cc:276] 0709 13:01:21.012119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.148542 (+1136423us) service_pool.cc:224] Handling call
0709 13:01:22.149166 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.242873 (+ 93707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.242919 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.245225 (+  2306us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:22.245273 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.245381 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.245503 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.250301  9449 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:22.254834  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73567.4us
W20260709 13:01:22.258473  8891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.039s
W20260709 13:01:22.258996  8891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.018s	sys 0.040s
I20260709 13:01:22.260387  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 621) took 2782 ms. Trace:
I20260709 13:01:22.261416  8747 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:22.256986 12539 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:22.290781  8890 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:22.265914  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.263379 12811 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.283377  8817 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
W20260709 13:01:22.284108  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45849.2us
I20260709 13:01:22.273460  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:22.285858  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 149989us
I20260709 13:01:22.306593  8819 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:22.267143  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:22.265708  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 339276us
I20260709 13:01:22.267256  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.283399 12742 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.270190 12722 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:22.274813 12682 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:22.270073 12637 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:22.344071  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2317) took 1665 ms (client timeout 1981 ms). Trace:
I20260709 13:01:22.346001  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.282606  8643 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 2 to 9, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.274536 12622 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:01:22.339921  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:22.348073 12725 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.340482  8820 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:01:22.323774  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 130130us
I20260709 13:01:22.274099 12654 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.362180  8890 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.351761  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.265269 12788 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:22.258498 12597 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.286391 12539 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:22.362254  9448 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:22.339160  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.368999  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:22.353544  8821 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.341892  8889 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:22.356169 12637 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.357916 12622 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:22.357962  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:22.353770  8815 rpcz_store.cc:269] 0709 13:01:20.678673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.725361 (+ 46688us) service_pool.cc:224] Handling call
0709 13:01:22.343702 (+1618341us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400082fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:22.343899 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2572744576}
W20260709 13:01:22.359014 12770 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.167s	user 0.008s	sys 0.045s
I20260709 13:01:22.358821 12811 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:22.266557 12850 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.360935  8896 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:22.360469  8643 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.152055  8741 rpcz_store.cc:276] 0709 13:01:18.055624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.817931 (+762307us) service_pool.cc:224] Handling call
0709 13:01:18.818345 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.818865 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.818909 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.836982 (+ 18073us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.140265 (+303283us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:19.140325 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.140476 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.774308 (+2633832us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:22.133766 (+359458us) raft_consensus.cc:1685] finished
0709 13:01:22.133845 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.134317 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":15613,"spinlock_wait_cycles":20531072,"thread_start_us":284441,"threads_started":1,"wal-append.queue_time_us":347701}
I20260709 13:01:22.337466  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 632) took 1557 ms. Trace:
I20260709 13:01:22.339625  8895 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:22.366361  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.367730  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.351810 12616 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.370797 12751 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:22.371665 12198 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.010s
I20260709 13:01:22.372224  9441 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:22.373092  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.373577  9449 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.373716  9440 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:22.374215  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.374990  9445 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.356917  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.379180  8823 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:22.403084  8740 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:22.272172 12837 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:01:22.260848  8891 rpcz_store.cc:276] 0709 13:01:19.477756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.679351 (+1201595us) service_pool.cc:224] Handling call
0709 13:01:20.679635 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.679963 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:20.726664 (+ 46701us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.721537 (+994873us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b18004978c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:01:22.259656 (+538119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.259767 (+   111us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:22.259884 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.260057 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.260271 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4280,"prepare.queue_time_us":46443,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":7295,"raft.queue_time_us":6380,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":299,"spinlock_wait_cycles":49419648,"thread_start_us":537681,"threads_started":1,"wal-append.queue_time_us":1351489}
I20260709 13:01:22.406309  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.351964 12722 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:22.412879  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.414091  9442 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.414790  8643 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 10, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:22.409801 12770 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.218s	user 0.009s	sys 0.046s
I20260709 13:01:22.416388 12788 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.417758 12616 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.418212 12682 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.419677  8892 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 7.
I20260709 13:01:22.421060  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.421905  8897 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:22.422650  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.471740  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.423883  9449 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 6. Current term is 8. Ops: []
I20260709 13:01:22.424450  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.425459 12654 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:22.426581  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.429509  8893 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.434496  8824 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:22.436487 12826 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.437474  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.437727 12751 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:22.438133  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 99229.2us
I20260709 13:01:22.438525 12667 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:22.439328 12803 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.440519 12764 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.481571  9444 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:22.481479 12764 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.375177 12810 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.482607  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.483291  9444 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.483425 12810 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.484390 12637 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.472947 12761 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.485049  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.485220 12761 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.485656  8891 raft_consensus.cc:2376] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8. Re-sending same reply.
I20260709 13:01:22.475639 12751 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:22.486629 12751 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.487468  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.488399 12616 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.489496  9447 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.490938  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.491659  9447 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:22.492614  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:22.493543  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67305.6us
W20260709 13:01:22.496054  8713 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.771s (SENT)
I20260709 13:01:22.496697 12847 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.442553  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.443372 12729 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.443547 12679 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:22.444185 12807 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:22.444756  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 616) took 2708 ms (client timeout 1586 ms). Trace:
I20260709 13:01:22.448788  8819 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:22.449405 12198 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:22.449707  8898 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.450148 12706 raft_consensus.cc:2764] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:22.450659 12783 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.451685  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.452337  8822 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:01:22.405124 12531 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:01:22.452852 12643 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.013s
I20260709 13:01:22.497098  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.486816  8748 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:22.506230  9443 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.454435  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.454653  8895 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:22.400750  8746 rpcz_store.cc:276] 0709 13:01:20.745830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.657458 (+911628us) service_pool.cc:224] Handling call
0709 13:01:21.657879 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.658531 (+   652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.658569 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.659138 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.134242 (+475104us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:22.243916 (+109674us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.254255 (+ 10339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.289142 (+ 34887us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.289217 (+    75us) raft_consensus.cc:1685] finished
0709 13:01:22.289233 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.303343 (+ 14110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":167947,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":27094,"raft.queue_time_us":235,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":352,"spinlock_wait_cycles":31488,"thread_start_us":583118,"threads_started":2,"wal-append.queue_time_us":390823}
W20260709 13:01:22.437493  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79742.5us
I20260709 13:01:22.459312  8896 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:22.463316  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 159754us
I20260709 13:01:22.463809  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:22.516754  9440 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.398581 12742 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.480286  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.380571 12622 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:22.379142 12539 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.475278  9442 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.482807  9441 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.498423  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:22.453222 12556 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:22.441522  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1717) took 1635 ms (client timeout 1873 ms). Trace:
I20260709 13:01:22.519716  8821 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:22.516714  8820 rpcz_store.cc:269] 0709 13:01:19.736011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.736138 (+   127us) service_pool.cc:224] Handling call
0709 13:01:22.444485 (+2708347us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b5400076a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:22.444663 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5987705088}
W20260709 13:01:22.523607  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.489059  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.489617 12314 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.525816  8817 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.547142  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.497187 12847 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.548224  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.571911  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.476531 12667 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.544297  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:22.551963  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.552867  9446 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.556600 12764 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.558115  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:22.559031 12862 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:01:22.559594 12781 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:22.560828 12622 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.514847 12729 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:22.527227  8745 rpcz_store.cc:269] 0709 13:01:20.806233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.859329 (+ 53096us) service_pool.cc:224] Handling call
0709 13:01:22.441113 (+1581784us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400021cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:22.441319 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118611200}
I20260709 13:01:22.601950  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.569919 12539 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.581068  9448 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:22.581274 12770 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:22.581694  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55523.9us
I20260709 13:01:22.582851  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.583420 12759 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.586655 12725 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:22.586944  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2264) took 1880 ms. Trace:
I20260709 13:01:22.587409 12233 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:22.611802  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.587724 12826 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:22.588326 12699 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:22.613080  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.578436 12847 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.613874  9448 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.497428 12806 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:01:22.613102 12826 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.616199 12826 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.617050 12826 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.617506 12622 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.590559  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:22.585601 12722 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.577661 12314 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:22.620718 12759 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.609513 12770 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:22.622083 12314 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.622828 12770 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.624235 12770 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.593045  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.508388 12788 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.521099 12679 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:22.604004 12781 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.604779 12729 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.606108 12742 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.473838 12850 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 13:01:22.606812 12811 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:22.545481 12198 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:22.545264 12803 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.673748 12667 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.673995 12198 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.675285 12803 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:22.611406 12616 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:22.676455  8893 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:22.589721  8889 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.613749 12699 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:22.620421 12826 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.610356  8892 rpcz_store.cc:276] 0709 13:01:20.706449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.132862 (+426413us) service_pool.cc:224] Handling call
0709 13:01:22.586701 (+1453839us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400047ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:22.586855 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2359138688}
I20260709 13:01:22.626083  8815 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.627401  9441 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.629469  8748 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:22.630838  8643 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:22.601051 12783 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.633622  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.649924  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.651929  8896 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:22.654740  8817 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:22.657239  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.661660 12759 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.661854  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.638669 12781 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:22.685648  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.665732 12761 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:22.665834 12679 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.669435 12433 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:22.670984  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80605.7us
I20260709 13:01:22.671314 12773 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:22.671377 12556 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.672550  8816 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:22.675542 12667 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.686913  9445 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:22.690585  9444 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:01:22.691569  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 133684us
I20260709 13:01:22.684674  9443 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:22.695394  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61961.3us
I20260709 13:01:22.702916  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:22.702193  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 111817us
I20260709 13:01:22.684679 12643 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.710769  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.712762  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.716215  9445 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.713917  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.679558 12699 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:22.717693  8713 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.868s (SENT)
I20260709 13:01:22.677271 12807 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:22.726835  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 637) took 1695 ms. Trace:
I20260709 13:01:22.727316  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.703553  8643 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.730432  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.733770  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:22.736364 12723 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:22.736768 12723 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:22.736888  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.737150 12723 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.738520  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.738807 12783 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.739295  8815 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 7.
I20260709 13:01:22.740053  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.710719 12722 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.728968  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:22.726583  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:22.742162  9447 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
W20260709 13:01:22.745005  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.743333  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.747717  9441 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:22.748687  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:22.750277  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:22.764205 12679 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.766219 12682 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.768047  8817 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:22.768628 12800 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.774981 12783 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:22.776033 12783 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.777047 12665 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:22.777536 12665 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:22.778504 12665 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.779793 12665 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.776402 12783 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:22.782202 12783 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.783274  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.784360  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.784971  9446 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:22.785698 12781 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.718561 12773 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:22.792270 12751 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.792742 12699 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.792284  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:22.795661  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:22.835359  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.836185  9440 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.797137  8889 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:22.769030 12800 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.740552 12531 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:22.802686  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 91776.5us
I20260709 13:01:22.803804 12806 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:22.804399  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.805318  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.804462  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:22.809849  9445 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:22.821481  9442 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:22.827319  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 105854us
I20260709 13:01:22.828676  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.862090  8824 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:22.862782  9441 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.829691  9447 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:22.833382  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.835436  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.864333  9444 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.750895  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.857141  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.858940 12531 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.865869  9446 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.867163  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.867756  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.868737  9441 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.869649  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:22.873392 12864 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:01:22.877313  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.878505  9447 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 8: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.878661  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.879777 12531 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.879719  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.881791  8745 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:22.883704  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.884379  9441 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:22.884444  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:22.883901  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 133216us
I20260709 13:01:22.883792  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.888103  9447 raft_consensus.cc:2376] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12. Re-sending same reply.
W20260709 13:01:22.885596  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.890036  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:22.891120  9446 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.892340  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:22.893481  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 181109us
I20260709 13:01:22.893707 12846 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:22.894166  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.894261 12846 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:22.895216 12846 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.895655 12837 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.896355 12837 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.896946  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1690) took 1109 ms. Trace:
I20260709 13:01:22.897363  8822 rpcz_store.cc:276] 0709 13:01:21.787089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.791011 (+  3922us) service_pool.cc:224] Handling call
0709 13:01:22.896725 (+1105714us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54003bf6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:22.896888 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094297472}
I20260709 13:01:22.898073  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.899435 12822 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.897461 12622 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:01:22.900162 12822 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:22.899392 12838 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.900795 12838 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.900833 12622 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.901314 12838 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.901106 12822 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.901611 12803 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:22.902119  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.902776  9449 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:22.901626  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.904014  9445 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:22.904206  9449 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 9 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:22.905064 12837 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.905933 12773 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.908295 12723 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:22.909412  8890 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 13:01:22.909759  8890 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:22.910023 12433 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.910642 12433 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.910923 12806 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:22.911701 12869 raft_consensus.cc:2764] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:22.912469 12314 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:22.912698  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:22.914840 12433 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:22.914760  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:22.916371 12870 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:22.916661  8822 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:22.917798 12781 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.918355 12838 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.921154  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.901439 12622 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:22.921509 12838 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:22.921923  8820 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:22.922262 12622 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.922791  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.923588  8821 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:22.923885 12622 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.925546 12850 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.926244 12850 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:22.927410 12850 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:22.927892  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:22.930709 12751 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.932267  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:22.935050 12871 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:01:22.937561 12198 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.938153 12198 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:22.938536 12198 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.943537  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.944190  8821 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.855541 12800 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.951167  8821 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:22.956043  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:22.957080  9440 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:22.893013  9446 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:22.886929  8643 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 2 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 6 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.959350  9449 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.960315  9441 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.914783 12699 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.947736 12847 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:22.730018  8749 rpcz_store.cc:276] 0709 13:01:21.031956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.181013 (+1149057us) service_pool.cc:224] Handling call
0709 13:01:22.189581 (+  8568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.256067 (+ 66486us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:22.479505 (+223438us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.726243 (+246738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.726293 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:22.726339 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.726659 (+   320us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.726779 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14536,"prepare.queue_time_us":90792,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":12529,"thread_start_us":298018,"threads_started":2}
I20260709 13:01:22.971381 12807 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.971776 12723 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:22.973222  9448 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:22.977859  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:22.979720  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:22.980661  8821 raft_consensus.cc:2368] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11.
I20260709 13:01:22.987327  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:22.992859  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:22.971443 12847 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:23.002429  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 70497.7us
I20260709 13:01:22.945223 12783 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.006291  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:23.009408 12807 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:23.010242 12872 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:01:23.013089 12723 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.013660  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:23.014302  8822 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:23.014775  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.016510  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.018033 12807 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:23.019172 12807 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.021355  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:23.022789  8892 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:23.022929 12539 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.024235  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.024878  8822 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.025632  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:23.026291  8890 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 7.
I20260709 13:01:23.027180  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:23.027969  8889 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:23.030380  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.012117  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.030627  8898 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:23.034772  9440 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:23.034793  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.036566  9447 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:23.017086  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 1997) took 2375 ms. Trace:
I20260709 13:01:23.049882  8821 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:23.053481  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125748us
I20260709 13:01:23.054620  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.055519  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 649) took 1243 ms. Trace:
I20260709 13:01:23.059037  8816 rpcz_store.cc:276] 0709 13:01:21.788943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.448083 (+659140us) service_pool.cc:224] Handling call
0709 13:01:22.672150 (+224067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.698991 (+ 26841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.031969 (+332978us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54000839d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:23.032136 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514568064}
I20260709 13:01:23.061091  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.062955  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.063978  9446 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:23.067025  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.078606 12783 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:23.083400  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 651) took 1293 ms. Trace:
I20260709 13:01:23.083846  8824 rpcz_store.cc:276] 0709 13:01:21.789655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.493769 (+704114us) service_pool.cc:224] Handling call
0709 13:01:22.494169 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.862718 (+368549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.083143 (+220425us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54000834d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:23.083327 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844155648}
I20260709 13:01:23.085644  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:23.086277  8748 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.088214  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:23.089076 12433 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:23.091020 12761 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.091621 12761 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:23.093856  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 132614us
I20260709 13:01:23.093654 12531 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.042128  8743 rpcz_store.cc:276] 0709 13:01:20.641960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.400779 (+758819us) service_pool.cc:224] Handling call
0709 13:01:22.445380 (+1044601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.446069 (+   689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.446103 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.446656 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.696195 (+249539us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:22.718304 (+ 22109us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.725422 (+  7118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.732417 (+  6995us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.885377 (+152960us) raft_consensus.cc:1685] finished
0709 13:01:22.885396 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.016663 (+131267us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400021cd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:23.016882 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32652,"prepare.queue_time_us":25569,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":15942,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":195,"spinlock_wait_cycles":2402290432,"thread_start_us":249103,"threads_started":1,"wal-append.queue_time_us":279320}
I20260709 13:01:23.096639 12811 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:23.097618 12811 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:23.098229 12237 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.098675 12237 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:23.098738 12811 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:23.099246 12811 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:23.099752 12237 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.100252 12811 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.061415  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:23.103641  8744 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:23.106276  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51781us
I20260709 13:01:23.106451  8643 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 7, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 7 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:23.109663 12867 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.156s	user 0.003s	sys 0.001s
I20260709 13:01:23.110850 12729 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.113054 12729 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:23.113828 12729 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.114202  8822 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:23.115231 12806 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:23.115783 12806 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.128036 12198 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.092590 12761 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.131086  8743 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:23.132869 12811 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:23.133069  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65732.9us
I20260709 13:01:23.133646  8749 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:23.133823  8748 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:23.107211  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.109169  8643 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 8, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:23.140403  9444 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:23.147431  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.150719  9443 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:23.163939  8744 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:23.170562  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.170881  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.147022 12237 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.174938  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.175824  8889 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:23.141893 12806 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:23.180801  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.181185  9415 leader_election.cc:400] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11.
I20260709 13:01:23.185010  9415 leader_election.cc:403] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:23.164137 12838 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:23.185791  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.163455 12759 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:23.186985  8898 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.195495  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.196221  8892 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:23.198943  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51744.9us
I20260709 13:01:23.177861 12729 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.178833 12531 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:23.205168  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.205626  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.200018  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.190223 12722 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:23.223058  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:23.225063  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.178042 12846 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:01:23.208807 12838 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.232897  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.237249  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:23.241554  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 430072us
W20260709 13:01:23.240981  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 133892us
W20260709 13:01:23.244195  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 180598us
I20260709 13:01:23.246378  8890 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:23.202252 12759 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.221354 12633 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11.
I20260709 13:01:23.206024 12800 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:23.250893  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.230887 12722 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.261442  8898 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:01:23.260075  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.264747  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.268498 12831 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.270195  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.274179 12868 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.275352  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.275467 12838 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.277232  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.278713 12803 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.279027  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.279455 12846 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.279541 12637 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:23.280220 12846 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.280954  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:23.283553  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:23.284052  8889 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:23.250125 12822 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:23.289151  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.294005  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:23.295017  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:23.294037  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.296343  9443 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:23.296581  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.298050  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.299437  9444 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:23.301409  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.301986  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:23.305243  9448 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.309525  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.311576  8890 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:23.314246  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.315493  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.290462 12822 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.331483  8896 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:23.334152  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:23.309088 12665 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.002s
I20260709 13:01:23.333078  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.341063  8889 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:23.305465 12807 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:23.318343  8863 connection.cc:256] RPC call timeout handler was delayed by -1.9899190840000074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
W20260709 13:01:23.343520  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60269.9us
I20260709 13:01:23.345372  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:23.346601  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49333.1us
W20260709 13:01:23.347190  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 52664us
I20260709 13:01:23.347369  9412 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.341352 12826 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:01:23.349247  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.346645  8744 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:23.362685  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:23.366117  8896 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.358424 12800 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:23.367311 12433 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:23.329972 12868 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.367987 12433 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.368279 12868 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.370112 12433 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:23.331277 12831 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.373404 12722 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.334504 12822 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.375988  8896 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:23.382324  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.373672 12831 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.353809 12846 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.385082 12846 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.386281 12831 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:23.387231 12846 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:23.387516 12831 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:23.387830 12846 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.388517 12846 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:23.330534 12773 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.338773 12637 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:23.356454 12826 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.342507 12723 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.342394 12198 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:23.391331  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:23.402484  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 69524.1us
W20260709 13:01:23.404183  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.404703  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.392027 12846 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.399348 12850 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:23.401445 12826 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:23.409273 12732 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:23.439021 12783 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.439021 12665 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.454068  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:23.445788 12869 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.443095 12806 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:23.438751 12807 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:23.471812  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.408965 12773 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:23.475952  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 91462.7us
W20260709 13:01:23.477625  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 133900us
W20260709 13:01:23.480665  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 131591us
I20260709 13:01:23.487285  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.466482 12783 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.469234 12850 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:23.494727  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:23.501986  8748 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:23.506738  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 159517us
I20260709 13:01:23.507508  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:23.511078  8892 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:23.480029 12314 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.503662  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.491556 12783 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.517848 12433 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:23.518054  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:23.518535  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:23.518894  8824 raft_consensus.cc:606] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.519120  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:23.519353  8824 raft_consensus.cc:614] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:23.520212 12800 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.520705 12800 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:23.521095 12800 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.523456 12868 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:23.524590 12822 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.519259  8815 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:23.519798 12826 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.518110  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:23.491137 12806 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:23.526418 12682 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:23.527012 12850 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.528465  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:23.528540 12622 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:23.576120 12826 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.576382 12622 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.503741 12531 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:23.529254 12314 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:23.533509 12800 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:23.543987  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 47582us
I20260709 13:01:23.537732 12807 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.544746 12869 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.539309 12773 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:23.541735 12539 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.004s
W20260709 13:01:23.542474  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.529333 12803 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.477079 12643 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:01:23.479892 12732 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.551330 12868 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:23.552363 12729 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.557488 12894 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:23.557528 12890 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:23.573064  8892 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:23.574416  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 84802.8us
I20260709 13:01:23.573702  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:23.565207 12759 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:23.593534  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:23.596890 12869 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.593824  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:23.625718  9444 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:23.599046  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:23.626384  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.613766  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.518834 12433 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:23.592955  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:23.507531 12838 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 13:01:23.627632  8889 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:23.630707  8815 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:23.632892  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.632892  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.634413  8821 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.617439 12237 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.640580  8824 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.640246  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:23.577404 12622 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.644562  8822 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:23.644273  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.653620  9443 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:23.642910  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 114593us
I20260709 13:01:23.644706  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:23.627223 12433 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:23.671939  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.630322 12869 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:23.686861  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 50142.7us
I20260709 13:01:23.565495 12846 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:23.701056  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.705456  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.706190  8819 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.708604  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:23.710312  8815 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:01:23.710709  8815 pending_rounds.cc:85] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 5
I20260709 13:01:23.631450 12781 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:01:23.681223 12759 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.715308  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.721086  8820 raft_consensus.cc:2410] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:01:23.723999  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.708071  8817 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.740686  8823 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:23.741608  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:23.747779  8822 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.660023 12900 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.660218 12894 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.684633 12803 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.684408 12682 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:23.702122 12890 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.710911  8815 pending_rounds.cc:107] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:01:23.762970  8815 raft_consensus.cc:2889] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.761053  8821 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:23.675863 12837 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:01:23.762959  8788 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:23.766587  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45240.3us
I20260709 13:01:23.771371  8889 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:01:23.772619  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.714793 12531 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:23.744671 12759 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.751825 12622 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.798817 12759 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.793859 12868 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:01:23.796437 12902 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.761725 12682 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:23.798264 12433 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.793592 12665 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:23.711385 12806 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.799001  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:23.753927 12846 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.805181  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:23.835460  9448 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 13:01:23.814450  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:23.814819  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.762781 12890 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.754418 12803 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.843472  8819 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:23.892818  8789 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.892s (SENT)
I20260709 13:01:23.818284  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:23.818424  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.817899  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:23.817153  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.815769  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:23.904793  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:23.909982  8824 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:23.895583  9448 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:23.815136  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:23.912583  8817 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:23.916131  8748 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:23.825912 12880 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.576s	user 0.004s	sys 0.000s
I20260709 13:01:23.823753 12539 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.817557  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.902288 12890 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.939240  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.907912 12803 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.836305 12781 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.819867 12751 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.958600 12539 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:23.981468 12751 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:23.817941 12902 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.982362 12539 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:23.982473 12751 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.983863 12751 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.826885 12847 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.984824 12732 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:23.946651 12723 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.985554 12732 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.986143 12732 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.986624 12889 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:23.988013 12889 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:23.989365  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.797044 12531 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.991981 12531 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:23.761497 12900 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.000097 12900 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:23.809866 12894 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:24.001739  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 261399us
I20260709 13:01:23.752468 12314 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:24.005599  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 210761us
W20260709 13:01:24.006233  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.006660  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.011361 12433 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:24.013217  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.943276  8741 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:23.980599  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:23.821957 12831 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 13:01:23.974172  8745 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:23.892398 12237 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:23.976596  8747 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:23.819288 12643 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.914597 12633 raft_consensus.cc:2764] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:23.984994 12847 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:23.944525  8822 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:24.020543  8815 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:24.021498  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 227766us
I20260709 13:01:23.828441 12759 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:23.819672 12665 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:24.035636  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.045949  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 447196us
I20260709 13:01:24.046911  8815 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:24.049749  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:23.984849 12902 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:24.063411  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:24.054535  8744 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:23.973407 12850 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.018563 12800 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.034694 12643 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:24.084100  8748 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:23.819408 12682 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.131758 12682 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.131886 12643 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.133384 12643 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.135629  8822 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:01:24.049669 12198 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.098692 12729 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:24.103766 12822 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.152889  8892 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:24.153983  8890 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:24.136096  8822 pending_rounds.cc:85] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 3
I20260709 13:01:24.154795  8822 pending_rounds.cc:107] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:01:24.155036  8822 raft_consensus.cc:2889] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:24.104831 12826 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 13:01:24.107647 12868 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.108577  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 334731us
I20260709 13:01:24.109606 12847 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.115598  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.116994  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.157670  8745 raft_consensus.cc:2368] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11.
I20260709 13:01:24.117776 12237 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.095281 12531 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:24.120317 12539 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.122108 12759 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.049199 12837 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.128484 12905 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.129576  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:24.130508  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.174172  8898 raft_consensus.cc:606] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:24.174700  8898 raft_consensus.cc:614] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:24.141798  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.097029  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:24.145677  8862 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:24.091631  8815 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.147820  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.149012  8743 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:24.158933 12822 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.161998  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:24.180984  8889 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.182328  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:24.162739 12783 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:01:24.161432 12826 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.169921  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.196936  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.169667 12729 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.179853  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.198442  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:24.193501  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.197347  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.198700  8748 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:24.186867  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 106036us
I20260709 13:01:24.200443  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.227530  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.180516 12633 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.241744  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:24.240794  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 143956us
I20260709 13:01:24.229696  8815 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.202117  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.244237  8745 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:24.170282 12847 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.202270  8747 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:24.227156  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 71047.5us
I20260709 13:01:24.156327 12850 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:01:24.233520  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.252239  8741 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:24.240638 12899 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:24.228246 12826 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.227806 12800 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:24.141105  8749 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:24.257761  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.162969 12868 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:24.250455  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.188097 12759 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.258747  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.259256  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.254114  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:24.245054  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.203320 12742 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.180522 12837 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.277729  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.245733 12761 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.189574 12822 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.279348 12800 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:24.279879  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.281469 12912 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.278469 12665 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.286281  8746 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:24.287699  8889 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
W20260709 13:01:24.288743  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 182036us
I20260709 13:01:24.290526  8817 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:01:24.307009  8817 pending_rounds.cc:85] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 9
I20260709 13:01:24.307345  8817 pending_rounds.cc:107] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:01:24.294498 12869 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.002s
I20260709 13:01:24.294608  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.296114  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:24.310237  8820 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.296221  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.296797  8822 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:24.310729  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:24.294240  8815 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:01:24.223515 12783 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.312602  8749 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:24.307596  8817 raft_consensus.cc:2889] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.201802 12889 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:01:24.308271  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:24.299559  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 93195.7us
I20260709 13:01:24.311602  8815 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 4
I20260709 13:01:24.311832  8743 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:24.185706 12905 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.289640 12850 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.267180 12531 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:01:24.285332 12759 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.312109  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:24.256759 12633 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.321458  8815 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:01:24.330304  8815 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.330845  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.331593  8746 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:24.331799  8890 raft_consensus.cc:2410] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:01:24.334033  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.335659  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 81701.6us
I20260709 13:01:24.339161  8743 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:24.340503  8743 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:01:24.341435  8743 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:24.342190  8891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:24.342744  8891 raft_consensus.cc:606] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:01:24.343089  8891 raft_consensus.cc:614] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:24.294359 12742 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.317152 12665 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:24.356118 12806 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:24.354601  8749 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:24.378063  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:24.325486 12637 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:01:24.302263 12761 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:24.402256  8787 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.514s (SENT) [suppressed 38 similar messages]
W20260709 13:01:24.411461  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66378.7us
I20260709 13:01:24.380262  8820 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:24.343039 12908 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:24.319092 12869 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.306079 12912 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.412909  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:24.432827  8745 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:24.432662  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:24.434597  8820 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:24.438503  8891 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 8: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:24.444144  8743 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:24.444633  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:24.445554  8824 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:24.451437  8892 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:24.411404 12916 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.455678  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.493765  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.436148 12699 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.496687  8818 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:24.494434  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62970.9us
I20260709 13:01:24.502024  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:24.498343  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.506297  8820 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 8: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:24.436954 12918 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.506637  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.511804  8822 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:24.512084  8824 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:01:24.510497  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.409860 12806 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:24.513401  8745 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:24.516235  8891 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:24.495434 12889 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.451969 12850 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:01:24.500299 12899 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.494277 12922 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:24.466490 12781 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.532382  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.442804 12822 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.535238  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:24.541419  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 228139us
I20260709 13:01:24.554095  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.507146 12699 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.530676 12837 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:24.557497  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.454010 12908 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.529624 12826 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.536115 12906 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.538707 12803 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.003s
I20260709 13:01:24.568354  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.445919 12800 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.539016 12868 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.572876  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.542133 12922 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:24.608083  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.548965 12531 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:24.666808  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.669203  8823 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:24.670148  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.670797  8823 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:24.500571 12916 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.664695  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.569854 12637 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.529922 12889 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.571717 12783 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:24.684118  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.666375 12682 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.002s
I20260709 13:01:24.681432  8823 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:24.579375 12732 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:24.686439  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.520171 12918 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.671490 12837 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.692225  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 190335us
I20260709 13:01:24.541642 12899 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:24.685631  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 134958us
I20260709 13:01:24.669113 12917 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.706745  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:24.707449  8891 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:24.698805 12539 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.003s
W20260709 13:01:24.726420  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.680478 12531 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:24.728216  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:24.732368  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.730542  8822 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:24.732851  8822 pending_rounds.cc:85] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 4
I20260709 13:01:24.733214  8822 pending_rounds.cc:107] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:24.733538  8822 raft_consensus.cc:2889] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.707288  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:24.665688 12803 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.737632  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:24.741230  9443 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.742219  8824 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:24.747380 12914 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
W20260709 13:01:24.749714 12925 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:01:24.750313 12531 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.751399  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:24.751828 12926 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:01:24.752241  8896 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:24.753011 12927 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:01:24.742918 12622 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:24.753222  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:24.753664 12924 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.754245 12924 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.759465  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.761256  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:24.766283 12783 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:24.768493  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.762246 12929 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:24.770843  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2350) took 1100 ms. Trace:
I20260709 13:01:24.771301  8816 rpcz_store.cc:276] 0709 13:01:23.670634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.684930 (+ 14296us) service_pool.cc:224] Handling call
0709 13:01:23.714328 (+ 29398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.242690 (+528362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.242727 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.257437 (+ 14710us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.457872 (+200435us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.457923 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.458036 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.770539 (+312503us) raft_consensus.cc:1685] finished
0709 13:01:24.770566 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.770735 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1978,"raft.queue_time_us":16007,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":10929,"thread_start_us":625735,"threads_started":2,"wal-append.queue_time_us":245623}
I20260709 13:01:24.768043  8713 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:24.764977 12936 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:24.775751 12783 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.779219  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:24.780367  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:24.781028 12922 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.782611 12761 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.754266  8896 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:24.788482  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:24.675802 12637 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.800762  9443 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:24.800958 12933 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.801745 12933 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.803920  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:24.804656 12933 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.805047  9444 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:24.727242 12838 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:24.584250 12906 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.794938 12783 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.816052  8896 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:24.814047  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 69466.6us
I20260709 13:01:24.733884 12633 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.818491  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:24.797863 12846 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.904s	user 0.010s	sys 0.002s
I20260709 13:01:24.846266  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:24.713784 12917 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.850646  9445 raft_consensus.cc:2376] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10. Re-sending same reply.
I20260709 13:01:24.852984  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:24.808192 12936 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:24.857115  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 115566us
I20260709 13:01:24.688637 12806 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.742874 12682 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:01:24.714826 12837 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.862313  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:24.870597  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.814875 12869 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.901475 12936 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:24.901971 12934 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:24.902771 12936 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.904767 12889 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:24.855249 12722 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.849910 12732 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.868451 12912 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.871301 12900 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:24.843905 12892 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.901822 12850 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:24.919562  8748 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:24.910153 12806 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.819049 12838 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:24.929407  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.934837  8747 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:24.864092 12759 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:24.911562 12847 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.004s
I20260709 13:01:24.917447 12811 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.909700 12916 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.910739 12237 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:01:24.912269 12732 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.939301  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.846266 12783 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:24.941223  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:24.920921 12900 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.919384 12933 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:01:24.964058  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:24.937943  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:24.916065 12807 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:01:24.966135  8748 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:24.926868 12924 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.915534 12722 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:24.943197 12890 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.941s	user 0.007s	sys 0.006s
I20260709 13:01:24.856745 12761 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:01:24.972260 12838 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.939733 12850 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:24.966527 12900 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.986965 12899 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:24.963654 12742 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:24.964136 12732 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.989424 12924 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:25.165329  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.977427 12890 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.168352 12890 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.947825 12811 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:25.169688  8818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2352) took 1498 ms (client timeout 1871 ms). Trace:
W20260709 13:01:25.170133  8818 rpcz_store.cc:269] 0709 13:01:23.671565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.684701 (+ 13136us) service_pool.cc:224] Handling call
0709 13:01:25.169494 (+1484793us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:25.169627 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722104704}
I20260709 13:01:25.171448  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:25.171676 12811 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:25.010939  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47013.8us
I20260709 13:01:25.173830 12905 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
W20260709 13:01:25.175150  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.175647  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.175940  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.175882  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.150007 12889 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.177768 12889 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.179378  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:25.180109  8817 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.180306 12889 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:25.150025  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1730) took 1231 ms. Trace:
I20260709 13:01:25.181771  8740 rpcz_store.cc:276] 0709 13:01:23.918309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.918463 (+   154us) service_pool.cc:224] Handling call
0709 13:01:23.918820 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.992663 (+ 73843us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:24.158921 (+166258us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.408471 (+249550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.408538 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:24.408601 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.899171 (+490570us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b1800299a40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2557 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:25.149745 (+250574us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b54004655d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:25.149808 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.149954 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83267,"prepare.queue_time_us":433228,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":78870,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9922,"spinlock_wait_cycles":273708288,"thread_start_us":307999,"threads_started":2,"wal-append.queue_time_us":263900}
I20260709 13:01:25.150957 12633 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.185231 12633 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.187808  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:25.188334  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.188792  8820 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:25.189253 12759 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.028441  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:25.228331  8889 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:25.230451  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:25.231120  8889 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.225805  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:25.233481 12892 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:25.225262  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.238315  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.146757 12937 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.173s	user 0.008s	sys 0.000s
W20260709 13:01:25.245358  8863 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.080s	sys 0.128s
W20260709 13:01:25.245842  8863 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.080s	sys 0.128s
W20260709 13:01:25.246181  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 580842us
W20260709 13:01:25.248134  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.249408  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:25.250854 12811 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.251298 12811 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.251670  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.252281  9444 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:25.252370  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:25.252709  9442 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:25.251761 12811 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.254319  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:25.130846 12838 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.255030 12838 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.255393  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.256313  9442 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:25.256784 12622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.021s	sys 0.099s
I20260709 13:01:25.257992  9443 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:25.259259  9446 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:25.260541  9449 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:25.261730  8889 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:25.261988  9440 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:25.262790 12751 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.263362 12751 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.264268  9448 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:25.132408  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2020) took 1126 ms. Trace:
I20260709 13:01:25.265801  8746 rpcz_store.cc:276] 0709 13:01:24.005827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.307966 (+302139us) service_pool.cc:224] Handling call
0709 13:01:25.132135 (+824169us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5400160dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:25.132327 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37701888}
I20260709 13:01:25.132594 12908 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:25.133044 12935 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.201s	user 0.000s	sys 0.008s
I20260709 13:01:25.135002 12837 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:24.965930 12916 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.137254 12900 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:25.142093  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.292104  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:25.138540 12722 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:25.156927 12850 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:25.145771 12906 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:25.157939 12643 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.024s	user 0.014s	sys 0.002s
I20260709 13:01:25.146633 12803 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:25.257156 12622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.021s	sys 0.099s
I20260709 13:01:25.159710  8749 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:25.172544  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.216615  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:25.298077  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:25.299036  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91176us
I20260709 13:01:25.205644 12933 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.204280 12912 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.187842 12633 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:25.189774  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.219251  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:25.280323 12892 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:25.190847  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.314109  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.314551  8815 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:25.202527  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.315006  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.199033  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.201200 12781 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.147785 12783 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:25.213625  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.201050 12917 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:25.016402 12699 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:25.224463  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48689.6us
I20260709 13:01:25.349138  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.350715  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.206157  8745 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:24.968045 12237 consensus_queue.cc:1243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:25.254122  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.262732  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:25.355803  8889 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:25.130652 12847 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.131923 12637 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:25.361346  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 662) took 1338 ms. Trace:
I20260709 13:01:25.361816  8749 rpcz_store.cc:276] 0709 13:01:24.022456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.451236 (+428780us) service_pool.cc:224] Handling call
0709 13:01:25.159417 (+708181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.297768 (+138351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.361139 (+ 63371us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400160dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:25.361267 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527894272}
I20260709 13:01:25.271685  8817 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
W20260709 13:01:25.272687  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 120147us
I20260709 13:01:25.161268  8891 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:25.278615  9447 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:01:25.154212 12433 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.142s	user 0.014s	sys 0.005s
I20260709 13:01:25.364244  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:25.288092  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.364789  8818 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:25.257120 12838 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:25.376544  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 122546us
I20260709 13:01:25.336097 12198 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.377638  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.377884 12198 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.379616  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.380468 12773 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:25.380728 12198 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:25.288003 12889 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.158813  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:25.299217  9444 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:25.422430  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 55666.1us
I20260709 13:01:25.210438  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1728) took 1394 ms. Trace:
W20260709 13:01:25.424199  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61563.3us
I20260709 13:01:25.302637  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.303035  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.304912  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.306229  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:25.425577  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.425928  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:25.307036  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:25.308220  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.426899  9440 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 9 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:25.309422  8892 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:25.153630 12918 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:25.312681  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:25.432860  8820 raft_consensus.cc:2410] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:01:25.205206 12930 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.508s	user 0.001s	sys 0.002s
I20260709 13:01:25.315567  8816 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:25.337433  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.346515  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.359055  8815 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
W20260709 13:01:25.271685  8819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.019s	sys 0.026s
W20260709 13:01:25.441172  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 134308us
W20260709 13:01:25.465437  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114857us
I20260709 13:01:25.365078  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:25.365188  9448 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:25.466325  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:25.307626 12622 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:25.311653 12906 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.374562  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:25.482710  8889 raft_consensus.cc:2368] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:25.375128 12847 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.381186 12773 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:25.356046 12781 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.362716 12917 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.382443 12933 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:25.383047 12732 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.787168 12732 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:25.386180  8818 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:25.788492  8822 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:25.789156 12952 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.214s	user 0.000s	sys 0.003s
I20260709 13:01:25.386750 12900 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:25.313941 12761 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.790544 12761 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.792872 12761 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:25.386974 12800 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.335754 12912 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:25.794441  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:25.795395 12933 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:25.796001 12912 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:25.335934 12921 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:01:25.309496 12922 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.287323 12751 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:25.423244  8895 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:25.426123  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45412.8us
W20260709 13:01:25.427871  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 269287us
I20260709 13:01:25.810269  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:25.425585  8740 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:25.428068  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:25.429634  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65108.2us
I20260709 13:01:25.425549  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 663) took 1400 ms. Trace:
I20260709 13:01:25.444228  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 668) took 1165 ms. Trace:
I20260709 13:01:25.466325  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.466328  9446 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:25.386780 12699 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:25.468096  8823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.025s	sys 0.037s
W20260709 13:01:25.468669  8819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.021s	sys 0.028s
I20260709 13:01:25.474218  8820 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:25.309113 12916 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.474701  8749 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:25.485158  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.424769  8742 rpcz_store.cc:276] 0709 13:01:23.816397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.817410 (+  1013us) service_pool.cc:224] Handling call
0709 13:01:23.817736 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.096629 (+278893us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:24.317116 (+220487us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.465738 (+148622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.465788 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.465827 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.178463 (+712636us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b180038dbe0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:25.210164 (+ 31701us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400200b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:25.210239 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.210385 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50113,"prepare.queue_time_us":366946,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":99319,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8611,"spinlock_wait_cycles":647559296,"thread_start_us":287395,"threads_started":2,"wal-append.queue_time_us":163075}
I20260709 13:01:25.485800  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:25.316037 12314 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:01:25.430269 12838 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:01:25.607190  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.580585  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:25.579697 12622 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.584066  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 657) took 1024 ms. Trace:
I20260709 13:01:25.440233 12237 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:01:25.441270 12433 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:01:25.464490 12682 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.430537 12783 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:25.563503  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 97315.2us
I20260709 13:01:25.576458  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:25.586114  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 233162us
I20260709 13:01:25.441097 12742 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:25.751310  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2298) took 1296 ms. Trace:
I20260709 13:01:25.569165  9445 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:25.566397 12908 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.468271 12531 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:25.575874 12930 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:25.464807 12665 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 13:01:25.440608 12889 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:25.674603  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:25.569139  8714 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.022s	sys 0.023s
I20260709 13:01:25.568693  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:25.666724 12950 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:01:25.784579 12936 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:01:25.781917  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 659) took 1212 ms. Trace:
I20260709 13:01:25.731019 12811 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:25.579309 12847 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:25.582007 12637 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:01:25.490595 12906 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:25.581946 12941 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:01:25.814276  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.109444 12905 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:26.109390 12936 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:01:26.111714 12783 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.113972  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.114639 12905 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:26.115156 12905 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.115888  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.116561  8749 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:26.117447  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.117990  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.118206  8749 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:26.118633  8745 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:26.119076  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.120136 12783 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.120217  8741 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
W20260709 13:01:26.122395  8823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.263s	user 0.029s	sys 0.048s
I20260709 13:01:26.121328  8748 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:26.121454 12936 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:26.121634  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2295) took 1770 ms. Trace:
I20260709 13:01:26.121901 12531 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:26.090086  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 279997us
I20260709 13:01:26.122346 12922 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:26.122331  8714 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.686s	user 0.028s	sys 0.030s
I20260709 13:01:26.027372 12933 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260709 13:01:25.932480  8895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2300) took 1418 ms (client timeout 1588 ms). Trace:
I20260709 13:01:26.123492  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.165521  9447 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:25.998704  8743 rpcz_store.cc:276] 0709 13:01:24.025231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.451294 (+426063us) service_pool.cc:224] Handling call
0709 13:01:24.451589 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.503412 (+ 51823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.503451 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.508405 (+  4954us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.910643 (+402238us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:24.910697 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.910764 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.425349 (+514585us) raft_consensus.cc:1685] finished
0709 13:01:25.425370 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.425493 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43413,"raft.queue_time_us":74873,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10488,"thread_start_us":401826,"threads_started":1,"wal-append.queue_time_us":799768}
I20260709 13:01:26.125660 12850 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:26.125882  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 545624us
I20260709 13:01:26.126205 12889 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.085709 12921 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.126827 12811 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:26.126875  8892 rpcz_store.cc:276] 0709 13:01:24.454858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.455281 (+   423us) service_pool.cc:224] Handling call
0709 13:01:25.751115 (+1295834us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54002099d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:25.751245 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855260928}
W20260709 13:01:25.994305 12960 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:01:26.029935  8891 raft_consensus.cc:2376] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:25.978340 12900 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:25.998210  8893 rpcz_store.cc:276] 0709 13:01:24.560165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.760925 (+200760us) service_pool.cc:224] Handling call
0709 13:01:24.789329 (+ 28404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:24.812888 (+ 23559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.812939 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:24.835616 (+ 22677us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.267796 (+432180us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:25.267872 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.268007 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.583814 (+315807us) raft_consensus.cc:1685] finished
0709 13:01:25.583840 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.583998 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9614,"raft.queue_time_us":24157,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10520,"thread_start_us":353106,"threads_started":1,"wal-append.queue_time_us":675719}
I20260709 13:01:26.129236  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2294) took 1795 ms. Trace:
I20260709 13:01:26.129917  8894 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:26.130506  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.131052  8889 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:26.010350  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 802) took 1200 ms. Trace:
W20260709 13:01:26.097347  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 99587.9us
I20260709 13:01:26.197760  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2070) took 1056 ms. Trace:
I20260709 13:01:26.198200  8889 rpcz_store.cc:276] 0709 13:01:25.141790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.501353 (+359563us) service_pool.cc:224] Handling call
0709 13:01:26.197698 (+696345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.132560  8898 rpcz_store.cc:276] 0709 13:01:24.569384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.768193 (+198809us) service_pool.cc:224] Handling call
0709 13:01:25.148063 (+379870us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:25.148322 (+   259us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:25.148866 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.148906 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:25.195809 (+ 46903us) log.cc:844] Serialized 78 byte log entry
0709 13:01:25.226860 (+ 31051us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b180047a040. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:01:25.487617 (+260757us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:25.778735 (+291118us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.780512 (+  1777us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.781446 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.781567 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.781634 (+    67us) raft_consensus.cc:1685] finished
0709 13:01:25.781648 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.781843 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":290609,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":24002,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":252,"spinlock_wait_cycles":100153088,"thread_start_us":260554,"threads_started":1,"wal-append.queue_time_us":398682}
I20260709 13:01:26.136027  8744 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:25.997514  8821 rpcz_store.cc:276] 0709 13:01:24.278890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.280503 (+  1613us) service_pool.cc:224] Handling call
0709 13:01:24.280808 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.281109 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.281136 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.281664 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.680663 (+398999us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:24.680722 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.680814 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.443971 (+763157us) raft_consensus.cc:1685] finished
0709 13:01:25.443999 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.444157 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73261,"raft.queue_time_us":54530,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":10289,"thread_start_us":398615,"threads_started":1,"wal-append.queue_time_us":588687}
I20260709 13:01:26.200423  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.201318  8821 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:26.136700  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.201994  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.202423  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.164670 12531 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.105448 12781 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.109687  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2052) took 1317 ms. Trace:
I20260709 13:01:26.123597 12742 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:26.122938 12906 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:26.144726  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 219377us
I20260709 13:01:26.144506 12976 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:26.094241  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 666537us
I20260709 13:01:25.924984  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:25.821542  9449 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:26.146637  8824 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:01:26.148402 12916 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.150030  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 671) took 1601 ms. Trace:
I20260709 13:01:26.017174 12800 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.092111  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.153018  8820 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:25.911492  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 98685.5us
I20260709 13:01:26.115144  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.155702  8749 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:26.087998 12773 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.002s
I20260709 13:01:26.223264  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2366) took 1014 ms. Trace:
I20260709 13:01:26.225617  9448 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:26.226310  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:26.226699 12800 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:26.227218 12800 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.227298 12807 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:26.227772 12807 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:26.185730 12936 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.229506 12237 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.229992 12237 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:26.230124 12936 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.230507 12237 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.230789 12732 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:26.227793  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.231769  8822 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:26.232198 12732 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:26.226707  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2047) took 1677 ms. Trace:
I20260709 13:01:26.214219  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:26.233980  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:26.234392  8893 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:26.224431  8819 rpcz_store.cc:276] 0709 13:01:24.549166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.549424 (+   258us) service_pool.cc:224] Handling call
0709 13:01:24.549885 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.550396 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:24.550435 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.564026 (+ 13591us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.149039 (+1585013us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:26.149081 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.149723 (+   642us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.149774 (+    51us) raft_consensus.cc:1685] finished
0709 13:01:26.149786 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.149972 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":542,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":8618,"thread_start_us":1468659,"threads_started":1,"wal-append.queue_time_us":580455}
I20260709 13:01:26.224876  8815 rpcz_store.cc:276] 0709 13:01:25.208754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.480735 (+271981us) service_pool.cc:224] Handling call
0709 13:01:25.548132 (+ 67397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.567280 (+ 19148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.567332 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.567941 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.046149 (+478208us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:26.046197 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.046271 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.223027 (+176756us) raft_consensus.cc:1685] finished
0709 13:01:26.223048 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.223207 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7439,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3687,"thread_start_us":353695,"threads_started":1,"wal-append.queue_time_us":528017}
I20260709 13:01:26.236434  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:26.234954  8824 pending_rounds.cc:85] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:01:26.216302  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2368) took 1007 ms. Trace:
I20260709 13:01:26.238335  8824 pending_rounds.cc:107] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:01:26.239002  8824 raft_consensus.cc:2889] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.158355  8890 rpcz_store.cc:276] 0709 13:01:24.351730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.351900 (+   170us) service_pool.cc:224] Handling call
0709 13:01:24.375982 (+ 24082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.465051 (+ 89069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.465093 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.503995 (+ 38902us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.874719 (+370724us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:24.874762 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.170309 (+295547us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b540046da54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:25.170353 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.121318 (+950965us) raft_consensus.cc:1685] finished
0709 13:01:26.121345 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.121576 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2095,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":42651,"spinlock_wait_cycles":175786880,"thread_start_us":308465,"threads_started":1,"wal-append.queue_time_us":856572}
I20260709 13:01:26.162513  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:26.167057  8895 rpcz_store.cc:269] 0709 13:01:24.514086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.514289 (+   203us) service_pool.cc:224] Handling call
0709 13:01:25.932248 (+1417959us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:25.932396 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048181632}
W20260709 13:01:26.171767  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.152803 12944 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.207329  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.187994  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.242071  8892 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:26.188064 12314 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.188891 12665 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:26.190261 12783 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.192066 12975 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:26.184424 12921 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.194047  9447 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:01:26.202716  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.910825  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.203872  8818 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:26.163444 12917 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.235152  8742 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:26.238454  8816 rpcz_store.cc:276] 0709 13:01:25.209383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.481481 (+272098us) service_pool.cc:224] Handling call
0709 13:01:25.549176 (+ 67695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.559423 (+ 10247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.559473 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.560109 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:01:25.941535 (+381426us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:25.941599 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.941700 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.216031 (+274331us) raft_consensus.cc:1685] finished
0709 13:01:26.216061 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.216232 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9770,"raft.queue_time_us":20434,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21441,"thread_start_us":255945,"threads_started":1,"wal-append.queue_time_us":547155}
I20260709 13:01:26.156993  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1723) took 1477 ms. Trace:
I20260709 13:01:26.243213  9440 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:26.243871  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2066) took 1297 ms. Trace:
I20260709 13:01:26.252699  8892 rpcz_store.cc:276] 0709 13:01:24.946129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.187737 (+1241608us) service_pool.cc:224] Handling call
0709 13:01:26.243809 (+ 56072us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.253463  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.234984  8896 rpcz_store.cc:276] 0709 13:01:24.792521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.854270 (+ 61749us) service_pool.cc:224] Handling call
0709 13:01:24.864487 (+ 10217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.130759 (+266272us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:25.548574 (+417815us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.109111 (+560537us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.109164 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:26.109232 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.109472 (+   240us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.109618 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55271,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":102239,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":112030976,"thread_start_us":842513,"threads_started":2,"wal-append.queue_time_us":523374}
I20260709 13:01:26.197782  9441 rpcz_store.cc:276] 0709 13:01:24.798266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.798407 (+   141us) service_pool.cc:224] Handling call
0709 13:01:24.798798 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.858241 (+ 59443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.858290 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.870526 (+ 12236us) log.cc:844] Serialized 26 byte log entry
0709 13:01:25.395262 (+524736us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:25.395332 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.593560 (+198228us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:25.593623 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.788362 (+194739us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:25.998154 (+209792us) raft_consensus.cc:1685] finished
0709 13:01:25.998176 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.998312 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2937,"raft.queue_time_us":8622,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":8622,"spinlock_wait_cycles":170862720,"thread_start_us":475610,"threads_started":1,"wal-append.queue_time_us":665587}
I20260709 13:01:26.245609  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:26.254084  8889 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:26.148516 12905 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.233254  8894 rpcz_store.cc:276] 0709 13:01:24.549560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.549774 (+   214us) service_pool.cc:224] Handling call
0709 13:01:26.129610 (+1579836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.196909 (+ 67299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.226505 (+ 29596us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54000843d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:26.226648 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3588527872}
I20260709 13:01:26.259016  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.252172  8823 rpcz_store.cc:276] 0709 13:01:24.679458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.695917 (+ 16459us) service_pool.cc:224] Handling call
0709 13:01:24.696198 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.696391 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:24.859636 (+163245us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.156529 (+1296893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.156600 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:26.156670 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.156760 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.156922 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":549077,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":75,"raft.queue_time_us":188,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4721,"thread_start_us":1459475,"threads_started":2,"wal-append.queue_time_us":717175}
W20260709 13:01:26.260079  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98338.7us
I20260709 13:01:26.259532  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.260844  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:26.261252  8896 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 8.
I20260709 13:01:26.196514  8897 rpcz_store.cc:276] 0709 13:01:24.333632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.339114 (+  5482us) service_pool.cc:224] Handling call
0709 13:01:24.369955 (+ 30841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.370432 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:24.370707 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.524488 (+153781us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.529806 (+  5318us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.537834 (+  8028us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.539154 (+  1320us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.539206 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.539305 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.763968 (+224663us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18001b02e0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f
0709 13:01:24.911544 (+147576us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b48001d01d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:01:25.204503 (+292959us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:26.129031 (+924528us) raft_consensus.cc:1685] finished
0709 13:01:26.129055 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.129181 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10000,"prepare.queue_time_us":37342,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":3390,"raft.queue_time_us":12925,"raft.run_cpu_time_us":5129,"raft.run_wall_time_us":390445,"spinlock_wait_cycles":42661376,"thread_start_us":141180,"threads_started":1,"wal-append.queue_time_us":190503}
I20260709 13:01:26.243557 12761 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:26.261247  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:26.218335 12889 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.287945  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 661) took 1345 ms. Trace:
I20260709 13:01:26.288095  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.293270  8896 rpcz_store.cc:276] 0709 13:01:24.916972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.259242 (+1342270us) service_pool.cc:224] Handling call
0709 13:01:26.261996 (+  2754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.276896 12751 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:26.294682  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.289175  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2054) took 1443 ms. Trace:
I20260709 13:01:26.295542  8891 rpcz_store.cc:276] 0709 13:01:24.846049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.364860 (+518811us) service_pool.cc:224] Handling call
0709 13:01:26.288956 (+924096us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54001806d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:26.289103 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1293724416}
I20260709 13:01:26.296025 12751 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:26.296988  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.296799 12751 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.297834  8891 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.299081  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:26.299726  8891 raft_consensus.cc:2376] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12. Re-sending same reply.
I20260709 13:01:26.247776 12433 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.250450 12773 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:26.223196 12900 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:26.247517 12936 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:26.290850  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.328995 12936 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.329425  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 665) took 1018 ms. Trace:
I20260709 13:01:26.329562 12936 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.329855  8890 rpcz_store.cc:276] 0709 13:01:25.310722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.248923 (+938201us) service_pool.cc:224] Handling call
0709 13:01:26.329361 (+ 80438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.331081  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.332376  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.308015  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 673) took 1376 ms. Trace:
I20260709 13:01:26.308950  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:26.333252  8890 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:26.334758  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:26.335343  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.335806  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:26.336400  8890 raft_consensus.cc:2410] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:26.300813  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.313534  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.313916  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.247342 12682 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:26.337266  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.337663  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2053) took 1517 ms. Trace:
I20260709 13:01:26.337901 12682 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.338035  8889 rpcz_store.cc:276] 0709 13:01:24.820748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.260897 (+1440149us) service_pool.cc:224] Handling call
0709 13:01:26.337607 (+ 76710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.338446  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.287278 12643 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:01:26.339063  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.228714 12807 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.340162  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.276324 12917 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.340626 12807 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.340865  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 666) took 1024 ms. Trace:
I20260709 13:01:26.172936 12922 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.342396 12917 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.342689 12922 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.344307 12921 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.344146 12922 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:26.317560 12751 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.344805 12921 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:26.255043 12906 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:26.323071  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.339232  8890 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:26.346509  8893 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:26.333534  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.301782  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:26.347298  8889 rpcz_store.cc:276] 0709 13:01:25.333148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.339918 (+1006770us) service_pool.cc:224] Handling call
0709 13:01:26.340812 (+   894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.333571  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.348443  8891 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:26.333822  8824 rpcz_store.cc:276] 0709 13:01:24.931264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.315862 (+384598us) service_pool.cc:224] Handling call
0709 13:01:25.357910 (+ 42048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.248498 (+890588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.307681 (+ 59183us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:26.307933 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813965056}
I20260709 13:01:26.289490 12811 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.358529  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.353806  8895 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:26.322149  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.302428 12916 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.356495  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.364076  8890 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:26.357089  8898 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:26.349036  8818 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:26.364701  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2059) took 1468 ms. Trace:
I20260709 13:01:26.254072 12944 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.225138 12742 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.369467  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2069) took 1228 ms. Trace:
I20260709 13:01:26.369920  8894 rpcz_store.cc:276] 0709 13:01:25.141505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.287544 (+1146039us) service_pool.cc:224] Handling call
0709 13:01:26.369406 (+ 81862us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.259189 12783 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.377619  8891 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:01:26.380519  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2056) took 1524 ms. Trace:
I20260709 13:01:26.302760 12908 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:01:26.380985  8891 rpcz_store.cc:276] 0709 13:01:24.856254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.357054 (+1500800us) service_pool.cc:224] Handling call
0709 13:01:26.377377 (+ 20323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.380170 (+  2793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.380450 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.381609  8890 rpcz_store.cc:276] 0709 13:01:24.896338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.359754 (+1463416us) service_pool.cc:224] Handling call
0709 13:01:26.360117 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.364450 (+  4333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.364650 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.357753 12811 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.314463 12892 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:01:26.407317  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.327580 12433 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.303413 12761 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.395529  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:26.357637 12933 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:01:26.256526 12314 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:26.409538  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 222441us
I20260709 13:01:26.446447  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:26.408892  8891 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:26.447355  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2063) took 1531 ms. Trace:
I20260709 13:01:26.447656  8891 rpcz_store.cc:276] 0709 13:01:24.915675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.390753 (+1475078us) service_pool.cc:224] Handling call
0709 13:01:26.408457 (+ 17704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.447111 (+ 38654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.447302 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.420691  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2369) took 1211 ms. Trace:
I20260709 13:01:26.376726 12924 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.433533 12773 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:26.444257  8894 raft_consensus.cc:2368] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:26.444518  8824 raft_consensus.cc:2376] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9. Re-sending same reply.
I20260709 13:01:26.315143 12889 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.406559 12811 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:26.453833 12969 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.319s	user 0.003s	sys 0.000s
I20260709 13:01:26.458037 12892 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:26.510972  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2305) took 1174 ms. Trace:
I20260709 13:01:26.461369 12900 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.511442  8898 rpcz_store.cc:276] 0709 13:01:25.336291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.207063 (+870772us) service_pool.cc:224] Handling call
0709 13:01:26.510801 (+303738us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:26.510909 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265860864}
I20260709 13:01:26.461521 12924 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.408820 12699 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:26.465859  8896 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:26.466571  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:26.481689  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 221016us
I20260709 13:01:26.498358 12783 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.498977  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.474774 12773 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.499869 12905 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:26.501365 12531 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.501791 12433 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:26.502641 12962 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.398s	user 0.000s	sys 0.003s
I20260709 13:01:26.506590 12761 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:26.506870  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.509567  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.510082 12889 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.510427  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:26.576000  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 324743us
I20260709 13:01:26.576399  8891 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:26.508997 12314 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.462319  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:26.582268  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 120056us
I20260709 13:01:26.582901  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:26.583709  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:26.583986  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.516150  8898 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:26.585855  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2067) took 1610 ms. Trace:
I20260709 13:01:26.589956  8898 rpcz_store.cc:276] 0709 13:01:24.975965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.512490 (+1536525us) service_pool.cc:224] Handling call
0709 13:01:26.512848 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.585573 (+ 72725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.585796 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:26.516819  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 267243us
I20260709 13:01:26.525864 12936 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.530589 12800 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.610044  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.610641  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.531838 12916 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:26.532403 12980 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:26.534004 12975 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:26.522929  8894 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.511457 12892 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.538440 12921 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:26.517984 12908 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.575340  8897 raft_consensus.cc:2376] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:26.588663  8891 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:26.626200  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.626739  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 662) took 1346 ms. Trace:
I20260709 13:01:26.627115  8891 rpcz_store.cc:276] 0709 13:01:25.280020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.581334 (+1301314us) service_pool.cc:224] Handling call
0709 13:01:26.588370 (+  7036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.626492 (+ 38122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.626691 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.544713 12622 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.628656  8745 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:26.520953  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:26.610742  8744 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.630132  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.530781 12237 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.611501  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:26.537231 12944 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.611501  8740 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:26.612262  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:26.640908  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.642064  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.642697  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:26.643998  8890 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.597764 12699 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.598300 12314 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.616560  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.617336  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:26.618722  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 663) took 1333 ms. Trace:
I20260709 13:01:26.544828 12643 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:01:26.622473  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.623083  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.627930  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:26.631444  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 110627us
I20260709 13:01:26.632350  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:26.641614  8747 raft_consensus.cc:2376] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12. Re-sending same reply.
I20260709 13:01:26.676448  8822 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:26.676957  8897 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:01:26.679366  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.449575  8817 rpcz_store.cc:276] 0709 13:01:25.209685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.779481 (+569796us) service_pool.cc:224] Handling call
0709 13:01:25.779864 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.783376 (+  3512us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:26.163057 (+379681us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.420096 (+257039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.420167 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:26.420232 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.420319 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.420617 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":508546,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":87,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":1471,"thread_start_us":620455,"threads_started":2}
I20260709 13:01:26.598014 12889 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:26.614151  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 278523us
I20260709 13:01:26.610100 12781 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.641193  8894 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:26.690205  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2302) took 1377 ms. Trace:
I20260709 13:01:26.690707  8894 rpcz_store.cc:276] 0709 13:01:25.313221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.617082 (+1303861us) service_pool.cc:224] Handling call
0709 13:01:26.640838 (+ 23756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.689890 (+ 49052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.690131 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.676894  8741 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:26.589181 12905 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:26.678974  8824 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.677858  8898 rpcz_store.cc:276] 0709 13:01:25.285854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.590818 (+1304964us) service_pool.cc:224] Handling call
0709 13:01:26.618101 (+ 27283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.618327 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.618364 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:26.618417 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:26.618459 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.618526 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.618657 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.684237  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.677858  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2303) took 1364 ms. Trace:
I20260709 13:01:26.701645  8897 rpcz_store.cc:276] 0709 13:01:25.313504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.626346 (+1312842us) service_pool.cc:224] Handling call
0709 13:01:26.675141 (+ 48795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.677605 (+  2464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.677797 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:26.616847 12947 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:26.627622 12892 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.685930  8893 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.650939 12433 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.692155  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:26.716082  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.717012  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84816us
W20260709 13:01:26.717777  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.693823  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 392030us
I20260709 13:01:26.696462  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.679039  8742 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:26.718919  8743 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.719504  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.719769  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.720395  8742 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:26.650002 12699 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.711194  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.721022  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.721112  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.739087 12699 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.737600  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:26.742347  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.737600  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.738816  8742 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:26.728214 12900 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:26.744278  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.743916  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.746016  8743 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:26.746438  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.748147  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.750383  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:26.750614  9447 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:26.751194  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.750432  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.750739  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.752682  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:26.753501  9447 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
W20260709 13:01:26.749137  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60769.5us
I20260709 13:01:26.751878  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:26.719760  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 253361us
I20260709 13:01:26.766408  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:26.679761 12314 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:26.627010 12975 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.689622 12781 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:26.780493  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88471.6us
I20260709 13:01:26.781088  8822 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:26.774016  9441 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 13:01:26.792116  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.797413  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:26.800323 12643 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.642064 12908 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.707955 12947 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.801335  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.765369  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:26.713665 12433 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.626890 12980 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.818984  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.802235 12761 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.834115  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:26.834690  8817 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:26.822533  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 60574.5us
I20260709 13:01:26.836406  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:26.805485  8741 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:26.804487  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:26.812413  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49557.3us
I20260709 13:01:26.803887  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:26.811638  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 186535us
W20260709 13:01:26.743803 12637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.025s
I20260709 13:01:26.862936  8824 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:26.873153  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.877106  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.841528  8822 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:26.878736  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.879410  8822 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:26.880436  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:26.772864 12665 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:26.891201 12978 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:01:26.891278 12665 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:26.794840 12781 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.892865 12811 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.894963 12811 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:26.897223 12936 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:26.897869 12761 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.887780  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:26.898097 12905 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:26.899626 12988 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.900094 12988 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.900606 12988 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:26.900986 12977 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:01:26.802116 12314 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.747995 12900 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:26.801548 12985 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:26.814100 12947 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:26.840138 12981 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
I20260709 13:01:26.772150 12987 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.898459 12761 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.902777  8892 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:26.836472 12433 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.823386 12889 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:26.890635  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.947916  8822 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:01:26.874477 12637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.002s	sys 0.025s
I20260709 13:01:26.915899  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:26.902401 12961 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:01:26.941793  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:26.965528  8898 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:26.944006  8895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 664) took 1652 ms (client timeout 1983 ms). Trace:
I20260709 13:01:26.944206  8889 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:26.900038 12732 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.966634  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:26.871878 12643 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:01:26.928275 12759 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:26.981892  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78347.3us
I20260709 13:01:26.936795 12987 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:27.053630  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.304975  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:26.966943  8895 rpcz_store.cc:269] 0709 13:01:25.291335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.247368 (+956033us) service_pool.cc:224] Handling call
0709 13:01:26.943744 (+696376us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400247f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:26.943934 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118699904}
I20260709 13:01:27.306758  8896 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
W20260709 13:01:27.304381  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 468262us
I20260709 13:01:27.325824  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:27.331444  8822 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:01:27.332600  9415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.001s	sys 0.033s
W20260709 13:01:27.333464  9415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.002s	sys 0.033s
I20260709 13:01:26.942736 12761 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:27.334460  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 578824us
W20260709 13:01:27.338346  8821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.019s	sys 0.024s
W20260709 13:01:27.343983  8821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.019s	sys 0.027s
I20260709 13:01:27.338258  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:27.071367  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52284.7us
W20260709 13:01:27.348754  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.349881 12665 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.145681  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:27.352749  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1730) took 1034 ms. Trace:
I20260709 13:01:27.353175  8824 rpcz_store.cc:276] 0709 13:01:26.318570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.018707 (+700137us) service_pool.cc:224] Handling call
0709 13:01:27.352663 (+333956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.145891 13005 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:27.358029 13005 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:27.358564 13005 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:27.362146  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:27.363032  8822 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
W20260709 13:01:27.366652  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 421288us
W20260709 13:01:27.367945  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.337826  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:27.021944 12742 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:27.314312 12732 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.392261  8822 pending_rounds.cc:85] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 8
I20260709 13:01:27.392675  8822 pending_rounds.cc:107] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:01:27.392935  8822 raft_consensus.cc:2889] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:27.375749  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.008977 12531 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.006469 12773 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.332593 12987 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.184063 12800 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.393697  8895 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:27.263330 12979 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.277937 12751 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.280021 12237 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.438324  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:27.357045 12975 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:27.399767 12682 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.039s	user 0.011s	sys 0.000s
I20260709 13:01:27.505100 12682 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:27.505498 12984 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:27.505955 12984 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:27.506328 12984 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.361817 13010 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.935514 12985 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.476066 12761 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.478796 12900 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:27.479882 12781 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.003s
W20260709 13:01:27.480305 12990 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.608s	user 0.000s	sys 0.001s
I20260709 13:01:27.486333  8895 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:27.487210 12976 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:27.401618 12732 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:27.492040 12993 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.614s	user 0.008s	sys 0.015s
W20260709 13:01:27.495683 12699 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.000s
I20260709 13:01:27.502137 13012 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.503082 12908 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.503270  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1726) took 1340 ms. Trace:
I20260709 13:01:27.583388  8816 rpcz_store.cc:276] 0709 13:01:26.162629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.256317 (+ 93688us) service_pool.cc:224] Handling call
0709 13:01:26.256654 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.513316 (+256662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.513347 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.516986 (+  3639us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.742118 (+225132us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:26.742238 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.742374 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.502978 (+760604us) raft_consensus.cc:1685] finished
0709 13:01:27.503010 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.503182 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28175,"raft.queue_time_us":86017,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":437397,"threads_started":2,"wal-append.queue_time_us":969230}
I20260709 13:01:27.281514 13015 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 13:01:27.454435 12922 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.509545 12314 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.559s	user 0.002s	sys 0.006s
W20260709 13:01:27.521579 12944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.026s	sys 0.076s
W20260709 13:01:27.562698  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.027s	sys 0.060s
W20260709 13:01:27.632958  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.028s	sys 0.060s
W20260709 13:01:27.633625  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 835003us
I20260709 13:01:27.462723 12979 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.469652 12751 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.561349 12781 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.576887 13005 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.561182 12976 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:27.579749 12998 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:01:27.581732 12985 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:27.608525 12988 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
I20260709 13:01:27.532706 13010 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.655999 12988 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.657763 13010 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.589144 13012 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.659430  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2377) took 1527 ms. Trace:
I20260709 13:01:27.638471 12314 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.660494 13012 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.661590 12314 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.659859  8819 rpcz_store.cc:276] 0709 13:01:26.131735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.236556 (+104821us) service_pool.cc:224] Handling call
0709 13:01:26.257214 (+ 20658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.257768 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:26.771635 (+513867us) log.cc:844] Serialized 14 byte log entry
0709 13:01:26.777366 (+  5731us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.658883 (+881517us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b18002cefc0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f6085d70 00007f90f6079293 00007f90f606ecea 00007f90f606f23a 00007f90f64d9a2a 00007f90f64e90bf 00007f90f64e9038
0709 13:01:27.659111 (+   228us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.659159 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.659207 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.659269 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.659376 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45740,"prepare.queue_time_us":153616,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1329,"raft.queue_time_us":3425,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":20506,"spinlock_wait_cycles":202370432,"thread_start_us":417886,"threads_started":1}
I20260709 13:01:27.662391  8817 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:27.662494 12781 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.663637 12988 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.610723 12908 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:27.664608  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 680) took 1352 ms. Trace:
I20260709 13:01:27.664453 12908 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:27.632869 12944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.026s	sys 0.077s
I20260709 13:01:27.653800 13023 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:27.666563 13023 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.667044 12751 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.651361 12682 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.668155 12682 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.670150 12682 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.670603 12908 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.670658 13005 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.674465  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1728) took 1436 ms. Trace:
I20260709 13:01:27.674870  8823 rpcz_store.cc:276] 0709 13:01:26.237757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.289717 (+ 51960us) service_pool.cc:224] Handling call
0709 13:01:26.290134 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.574801 (+284667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.574830 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.583871 (+  9041us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.912258 (+328387us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:26.912478 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.912751 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.674249 (+761498us) raft_consensus.cc:1685] finished
0709 13:01:27.674273 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.674399 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3158,"raft.queue_time_us":52662,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"thread_start_us":525017,"threads_started":2,"wal-append.queue_time_us":1046956}
W20260709 13:01:27.678356  8892 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 669) took 2192 ms (client timeout 1916 ms). Trace:
I20260709 13:01:27.584067 12699 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.651171 12976 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.679186  8824 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:27.678833  8892 rpcz_store.cc:269] 0709 13:01:25.486184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.262815 (+776631us) service_pool.cc:224] Handling call
0709 13:01:27.678130 (+1415315us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54003bcc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:27.678261 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280730880}
I20260709 13:01:27.679934  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1737) took 1080 ms. Trace:
I20260709 13:01:27.664956  8817 rpcz_store.cc:276] 0709 13:01:26.312549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.872798 (+560249us) service_pool.cc:224] Handling call
0709 13:01:27.664402 (+791604us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b540024a4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:27.664549 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804563712}
I20260709 13:01:27.680302  8823 rpcz_store.cc:276] 0709 13:01:26.599182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.676400 (+1077218us) service_pool.cc:224] Handling call
0709 13:01:27.676859 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.677042 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.677072 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.677127 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.677160 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.677228 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.679881 (+  2653us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.399695 12889 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.684729  8817 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:01:27.684921  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 309115us
I20260709 13:01:27.685761  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1738) took 1073 ms. Trace:
I20260709 13:01:27.686445  8817 rpcz_store.cc:276] 0709 13:01:26.612720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.681183 (+1068463us) service_pool.cc:224] Handling call
0709 13:01:27.681504 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.685410 (+  3906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.685668 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:27.689002  8894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.008s	sys 0.020s
W20260709 13:01:27.680434  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 354827us
W20260709 13:01:27.692579  8894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.009s	sys 0.020s
I20260709 13:01:27.669085 13023 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.646283 12900 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.695441  8862 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.002s	sys 0.014s
I20260709 13:01:27.699381  8787 leader_election.cc:400] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:27.704017  8787 leader_election.cc:403] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:27.700611  8862 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.002s	sys 0.014s
W20260709 13:01:27.718611  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 883880us
I20260709 13:01:27.724902  8817 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:27.679711 12800 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:27.633158 12922 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.729664  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:27.719405  9447 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.000s	sys 0.011s
W20260709 13:01:27.738845  9447 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.000s	sys 0.011s
W20260709 13:01:27.705713 12838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.006s	sys 0.011s
W20260709 13:01:27.701795 12921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.009s	sys 0.006s
I20260709 13:01:27.721725 12900 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.772434  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2088) took 1005 ms. Trace:
I20260709 13:01:27.810292  8898 rpcz_store.cc:276] 0709 13:01:26.768310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.833402 (+ 65092us) service_pool.cc:224] Handling call
0709 13:01:27.772247 (+938845us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b5400248954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:27.772373 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200908672}
I20260709 13:01:27.727833 12742 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:27.777055  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.781452 12622 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:27.698796 12980 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:27.783712  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 83451.4us
I20260709 13:01:27.786775 12908 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.792270  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2380) took 1642 ms. Trace:
W20260709 13:01:27.788086  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58911.3us
I20260709 13:01:27.794907  8787 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:27.823536  8820 rpcz_store.cc:276] 0709 13:01:26.149634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.247470 (+ 97836us) service_pool.cc:224] Handling call
0709 13:01:26.247811 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.248069 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:26.248099 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.257526 (+  9427us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.557576 (+300050us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:26.557638 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.557735 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.491421 (+933686us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:27.782026 (+290605us) raft_consensus.cc:1685] finished
0709 13:01:27.782055 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.792191 (+ 10136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18160384,"thread_start_us":218815,"threads_started":1,"wal-append.queue_time_us":516247}
I20260709 13:01:27.824890  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:27.789579 12838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.007s	sys 0.011s
I20260709 13:01:27.789974 12807 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.793365 12947 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.002s
I20260709 13:01:27.828142 12924 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.828258 12947 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.829559  9440 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:27.829403 13021 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:27.830056 13021 consensus_queue.cc:1243] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:27.830459 12947 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:27.830869  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 844) took 1081 ms. Trace:
I20260709 13:01:27.831185 12807 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.831278  9440 rpcz_store.cc:276] 0709 13:01:26.749388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.750179 (+   791us) service_pool.cc:224] Handling call
0709 13:01:27.830663 (+1080484us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:27.830784 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479018880}
I20260709 13:01:27.832386 12900 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.832834 12997 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.833035 12986 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.218s
I20260709 13:01:27.833485 12807 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:27.833405 12997 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.702517 13019 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:01:27.835619 12997 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.835909 12732 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:27.836437 13017 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:01:27.808142 12921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.010s	sys 0.007s
I20260709 13:01:27.826478  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:27.846668  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:27.827708 12781 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:27.838485  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 849) took 1079 ms. Trace:
I20260709 13:01:27.839174  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2379) took 1707 ms. Trace:
I20260709 13:01:27.851199  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2064) took 2919 ms. Trace:
I20260709 13:01:27.851606  8890 rpcz_store.cc:276] 0709 13:01:24.931400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.390789 (+1459389us) service_pool.cc:224] Handling call
0709 13:01:26.391125 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.644466 (+253341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.850974 (+1206508us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540032b254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:27.851144 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581317888}
I20260709 13:01:27.851419  8821 rpcz_store.cc:276] 0709 13:01:26.132176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.236998 (+104822us) service_pool.cc:224] Handling call
0709 13:01:26.237271 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.625748 (+388477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.625782 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.643161 (+ 17379us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.344624 (+701463us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:27.344665 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.344735 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.838950 (+494215us) raft_consensus.cc:1685] finished
0709 13:01:27.838977 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.839125 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47326,"raft.queue_time_us":81701,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":26917,"thread_start_us":903584,"threads_started":2,"wal-append.queue_time_us":937986}
I20260709 13:01:27.782999 12800 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.850467 12781 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.868930 12781 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.869598  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.871104  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.871243 12800 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:27.872287  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.855036  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.860841 12979 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:27.882874  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 58471.7us
I20260709 13:01:27.850559  9444 rpcz_store.cc:276] 0709 13:01:26.759575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.760535 (+   960us) service_pool.cc:224] Handling call
0709 13:01:26.763844 (+  3309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.143400 (+379556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.143429 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.260833 (+117404us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.603480 (+342647us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:27.603518 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.603609 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.838181 (+234572us) raft_consensus.cc:1685] finished
0709 13:01:27.838209 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.838410 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34320,"raft.queue_time_us":448496,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":60976,"thread_start_us":611286,"threads_started":2,"wal-append.queue_time_us":504301}
I20260709 13:01:27.863010 12682 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.864219 12984 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.865474 12742 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:27.867581  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:27.854390  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:27.873517  8895 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:27.874416  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.820266 12622 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:27.876185  8892 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:27.879642  8789 leader_election.cc:336] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.514s (SENT)
I20260709 13:01:27.885102  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:27.884804 12994 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:27.808326 12988 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:01:27.890273  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:27.890432  8821 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:27.899884  8635 maintenance_manager.cc:643] P 5b6e9771cda34fa68ec7172c32ecd31b: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.525s	user 0.202s	sys 0.098s Metrics: {"bytes_written":30216,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":587903,"dirs.run_cpu_time_us":563,"dirs.run_wall_time_us":13950,"drs_written":1,"lbm_read_time_us":446,"lbm_reads_lt_1ms":4,"lbm_write_time_us":447141,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":8,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":471170,"threads_started":1,"update_count":13}
I20260709 13:01:27.887720  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:27.903575  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:27.907473  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:27.903941  8898 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:27.906805  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:27.930544  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:27.876564 13019 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:27.937001  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.841984 12980 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 13:01:27.902362 12892 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.129s	user 0.010s	sys 0.002s
W20260709 13:01:27.971017  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 91449.2us
W20260709 13:01:27.979494  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.934708 13023 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:27.874472 12314 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:27.947088  8822 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.027s
I20260709 13:01:28.004428  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:27.932857 12986 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.003170  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.025724  8743 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
W20260709 13:01:27.976475  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46050.9us
W20260709 13:01:28.006966  8789 leader_election.cc:336] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.529s (SENT)
I20260709 13:01:28.029486  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.007098  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.030805  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.007782  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:27.932706 12924 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:27.929262 12988 consensus_queue.cc:1243] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:27.929898 12237 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:28.021806  8822 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.000s	sys 0.028s
I20260709 13:01:28.022647  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.035979  8740 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:28.023898  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.036913  8746 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:28.037250  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 670) took 2548 ms. Trace:
I20260709 13:01:28.024753  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.037142  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.030750  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.038012  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
W20260709 13:01:27.931087 12433 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.925s	user 0.006s	sys 0.004s
I20260709 13:01:27.973402 12761 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.029827  8745 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:28.031198  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.037693  8894 rpcz_store.cc:276] 0709 13:01:25.488927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.691896 (+1202969us) service_pool.cc:224] Handling call
0709 13:01:26.718164 (+ 26268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.718443 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:27.693144 (+974701us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.036943 (+343799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.036993 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:28.037033 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.037091 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.037201 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5669,"prepare.queue_time_us":1186160,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"thread_start_us":1239669,"threads_started":2}
I20260709 13:01:28.030243  8820 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:27.904233 12682 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:27.903501 12989 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 13:01:28.053562  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.054207  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.054484  8820 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:28.054762  8746 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.043344  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.055385  8749 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:28.007988 12800 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.007098 13021 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:28.032549 12986 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.066964 12986 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.020634 12892 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.058501  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.069734  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.070292  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.071173  8741 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:01:28.059350  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.012815 13037 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:28.065209 12682 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.065780 13021 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.039598 13019 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:28.085599 13038 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:28.084955 13014 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:28.044535 12924 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:28.091464  8746 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:28.045099 12732 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
W20260709 13:01:28.095670  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 64573.1us
I20260709 13:01:28.103066  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.086987 12781 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.097150  8636 maintenance_manager.cc:419] P 5b6e9771cda34fa68ec7172c32ecd31b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9522 bytes on disk
I20260709 13:01:28.045360 12699 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.055166 12751 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
W20260709 13:01:28.098129 12987 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.004s
I20260709 13:01:28.104499 13002 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.044651 12433 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.092125 13019 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.107120 12759 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:28.107774 13028 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:28.108053 13030 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.109095  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.081315 13037 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.041474 12237 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:28.106954 13000 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:01:28.099440 12936 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:28.104660 13005 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.105093 12924 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:28.161904  8892 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:28.156419 13019 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:28.163473  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 156570us
I20260709 13:01:28.119488 12947 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:28.168640  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:28.169741  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66799.2us
I20260709 13:01:28.080571 12800 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.184685  8635 maintenance_manager.cc:643] P 5b6e9771cda34fa68ec7172c32ecd31b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.029s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":447,"lbm_reads_lt_1ms":4}
I20260709 13:01:28.143901 13002 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.109164 12986 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.186687 12924 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:28.196372  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.202831  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:28.201880  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 50978.2us
I20260709 13:01:28.165072 13037 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.202561  8895 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:28.215878  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:28.174598 12944 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.158737 12759 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.193423 12922 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:28.151010 12987 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.217639  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.221045  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.168308 13030 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.210155 13002 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.229115  8892 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:28.233613  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.240268  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:28.228089 12976 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.001s
I20260709 13:01:28.186908 13005 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.243268  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.238458  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 848) took 1485 ms. Trace:
I20260709 13:01:28.243882  8892 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:28.244664  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.219964 12936 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:28.244064  9446 rpcz_store.cc:276] 0709 13:01:26.752721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.752829 (+   108us) service_pool.cc:224] Handling call
0709 13:01:26.753109 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.981831 (+228722us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:27.131801 (+149970us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.696274 (+564473us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.704348 (+  8074us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.704833 (+   485us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:27.704882 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.704980 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.238257 (+533277us) raft_consensus.cc:1685] finished
0709 13:01:28.238275 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.238405 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47587,"prepare.queue_time_us":122399,"prepare.run_wall_time_us":47,"raft.queue_time_us":364291,"raft.run_cpu_time_us":5263,"raft.run_wall_time_us":222859,"spinlock_wait_cycles":75379584,"thread_start_us":530193,"threads_started":2,"wal-append.queue_time_us":715631}
W20260709 13:01:28.236670  8789 leader_election.cc:336] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.620s (SENT)
I20260709 13:01:28.241333  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.216005 12984 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.247961  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.249692  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.248538  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.250694  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.250993  8892 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.235787 12922 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.228508 13032 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.226999 12989 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:01:28.259301  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.242755 13016 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.248610 12980 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.252130 12994 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.273204  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.281533  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:28.270177 12984 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.282464  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.256565 13032 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:28.272583 13041 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:01:28.289335  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 850) took 1523 ms. Trace:
I20260709 13:01:28.277626 12838 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:28.256233 12889 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:28.298200 12237 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.299536 12986 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:28.283869 13044 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:01:28.301101 12781 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:28.301819 13042 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:28.302861 13005 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.278656 12980 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.306407 12980 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.307235 13022 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:28.308060 13036 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.310026  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.311234  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.314023 12637 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:28.314551  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2388) took 1664 ms. Trace:
I20260709 13:01:28.315099  8820 rpcz_store.cc:276] 0709 13:01:26.650446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.084378 (+1433932us) service_pool.cc:224] Handling call
0709 13:01:28.084840 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:28.095796 (+ 10956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.095830 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:28.096432 (+   602us) log.cc:844] Serialized 52 byte log entry
0709 13:01:28.165330 (+ 68898us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.165388 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.165491 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.314325 (+148834us) raft_consensus.cc:1685] finished
0709 13:01:28.314344 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.314499 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2871,"raft.queue_time_us":81640,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14310,"thread_start_us":68414,"threads_started":1,"wal-append.queue_time_us":150427}
I20260709 13:01:28.314599 12637 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:28.316787 13039 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:28.317256 13039 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:28.317579 12637 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.318063 13039 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.317975  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1735) took 1856 ms. Trace:
I20260709 13:01:28.319335 13039 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.319463  8816 rpcz_store.cc:276] 0709 13:01:26.461327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.607830 (+1146503us) service_pool.cc:224] Handling call
0709 13:01:27.608275 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.610843 (+  2568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.610891 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.611454 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.974321 (+362867us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:27.974377 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.005752 (+ 31375us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.317771 (+312019us) raft_consensus.cc:1685] finished
0709 13:01:28.317790 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.317900 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2073,"raft.queue_time_us":54814,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"thread_start_us":319263,"threads_started":1,"wal-append.queue_time_us":375276}
I20260709 13:01:28.316797  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.249980 12976 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.321228  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.311756  8821 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:28.322690  8894 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:28.323246  8816 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.324086  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.325659  8821 raft_consensus.cc:2410] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:28.270804 13016 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.324541  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.286262  8892 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:28.330729  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2378) took 2198 ms. Trace:
I20260709 13:01:28.272207 13020 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:28.331101  8815 rpcz_store.cc:276] 0709 13:01:26.131955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.236600 (+104645us) service_pool.cc:224] Handling call
0709 13:01:26.236899 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.254241 (+ 17342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.254274 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.258267 (+  3993us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.597344 (+339077us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:26.597399 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.597494 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.316076 (+1718582us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b24000e9828. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f3420fb0 00007f90f6097488 00007f90f6097be8 00007f90f606f398 00007f90f606a76a 00007f90f6069bb6
0709 13:01:28.321363 (+  5287us) raft_consensus.cc:1685] finished
0709 13:01:28.321386 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.330668 (+  9282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6893,"raft.queue_time_us":23138,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":25923,"spinlock_wait_cycles":14434048,"thread_start_us":240369,"threads_started":1,"wal-append.queue_time_us":491007}
I20260709 13:01:28.331701  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.285661 12989 consensus_queue.cc:1243] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:28.335944  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.338548  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.339206  8894 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:28.293226 12314 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.288940 13038 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:28.278132 13033 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:28.294404 13032 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.346179  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.283336 12994 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.289047 13031 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:28.329872 13026 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:28.364886  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.366709  8741 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:28.271281 12922 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.330049 13016 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.367650  8820 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:28.352432  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.376982  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.346843  8892 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.377228  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.382869  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.383093  8749 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.343858 13005 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:28.344446 12314 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.386404  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.387926  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.394742  8748 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:28.343691 13038 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.345722 13033 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.342759 13021 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.368633 13031 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.344801 12433 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:28.402333  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.403735  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.412091  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.412730  8748 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10. Re-sending same reply.
I20260709 13:01:28.375128 12994 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.406378  8894 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:28.407591  8747 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:28.381453 13026 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:28.340001 13020 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:28.404677 13045 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:01:28.397329 12314 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.421950 12922 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.413625  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.403352 12979 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:28.448488 12979 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.450500  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 673) took 2207 ms. Trace:
W20260709 13:01:28.412844 13056 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:01:28.450965  8897 rpcz_store.cc:276] 0709 13:01:26.243473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.710890 (+467417us) service_pool.cc:224] Handling call
0709 13:01:26.780080 (+ 69190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.572680 (+792600us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:27.903582 (+330902us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.112249 (+208667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.112303 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:28.112350 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.449450 (+337100us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b1800043900. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419
0709 13:01:28.450234 (+   784us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b540045abd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:28.450303 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.450441 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3984,"prepare.queue_time_us":439753,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":369,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":31718,"spinlock_wait_cycles":1870994432,"thread_start_us":538284,"threads_started":2,"wal-append.queue_time_us":399811}
I20260709 13:01:28.452467  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1731) took 2094 ms. Trace:
I20260709 13:01:28.452778  8822 rpcz_store.cc:276] 0709 13:01:26.358608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.380700 (+1022092us) service_pool.cc:224] Handling call
0709 13:01:27.381131 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:28.035837 (+654706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.035868 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:28.036423 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 13:01:28.208074 (+171651us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.208194 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.209858 (+  1664us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.452277 (+242419us) raft_consensus.cc:1685] finished
0709 13:01:28.452296 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.452416 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":255354,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":665138,"threads_started":2,"wal-append.queue_time_us":173259}
W20260709 13:01:28.417945 12908 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.002s
I20260709 13:01:28.431648  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:28.462990  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.431943  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.432137  8749 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:28.433666 13055 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:01:28.433887  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 846) took 1684 ms (client timeout 1974 ms). Trace:
I20260709 13:01:28.465183  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.295622  9447 rpcz_store.cc:276] 0709 13:01:26.766206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.767921 (+  1715us) service_pool.cc:224] Handling call
0709 13:01:26.769141 (+  1220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.794021 (+1024880us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:27.978945 (+184924us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.113214 (+134269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.113766 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.113924 (+   158us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:28.114145 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.114415 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.114498 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.289095 (+174597us) raft_consensus.cc:1685] finished
0709 13:01:28.289115 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.289260 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6664,"prepare.queue_time_us":459251,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":497068,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":88530,"thread_start_us":1221859,"threads_started":3,"wal-append.queue_time_us":240800}
I20260709 13:01:28.466593  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.437603 12800 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.475522 13030 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.443915  8892 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:28.476883  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.477509  8892 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:28.478343  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.478874  8892 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.347760 12622 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.482561 13031 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.482935 12622 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:28.445361  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.410053 13033 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.428723 13036 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:28.447551 13007 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.124s	user 0.004s	sys 0.000s
I20260709 13:01:28.454515  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 815) took 1705 ms. Trace:
I20260709 13:01:28.405419 13005 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:28.455564  8822 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:28.439823  8895 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:28.500600  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.405237 13038 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.430958  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.452924  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.457795  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.505820  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 685) took 1161 ms. Trace:
I20260709 13:01:28.506145  9446 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.461403  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.449123 12979 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.508661  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.467110  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.509254  8748 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.510852 13005 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.513545  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.468044  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.469887  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.471355  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.473062 12976 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:28.473150  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.516249  9447 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:28.482846  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.446218 13020 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:28.517323  9441 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:28.484664  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.492291  8890 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:28.520236  9445 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.496284  8892 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:28.464352 12908 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.497044  9448 rpcz_store.cc:276] 0709 13:01:26.749067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.749205 (+   138us) service_pool.cc:224] Handling call
0709 13:01:26.749579 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:26.917046 (+167467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.917076 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:27.769182 (+852106us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b2c000ebf38. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:27.771080 (+  1898us) log.cc:844] Serialized 52 byte log entry
0709 13:01:28.008986 (+237906us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:28.009045 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.009169 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.454215 (+445046us) raft_consensus.cc:1685] finished
0709 13:01:28.454236 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.454455 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":354455,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":93914,"spinlock_wait_cycles":28650880,"thread_start_us":404458,"threads_started":2,"wal-append.queue_time_us":460502}
I20260709 13:01:28.469015 12783 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:28.465559  9449 rpcz_store.cc:269] 0709 13:01:26.750043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.750154 (+   111us) service_pool.cc:224] Handling call
0709 13:01:28.433688 (+1683534us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54004a4554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:28.433827 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973732096}
I20260709 13:01:28.517269  8897 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:28.519204  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47972.2us
I20260709 13:01:28.521554  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.522179  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.542858  9447 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:28.521250  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.544488  8890 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:28.526257  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2385) took 1904 ms. Trace:
I20260709 13:01:28.535027  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2061) took 3623 ms. Trace:
I20260709 13:01:28.545461  8893 rpcz_store.cc:276] 0709 13:01:24.911265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.364506 (+1453241us) service_pool.cc:224] Handling call
0709 13:01:26.387730 (+ 23224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.707701 (+319971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.534766 (+1827065us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400249354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:28.534931 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685537024}
I20260709 13:01:28.545132  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.545015  8817 rpcz_store.cc:276] 0709 13:01:26.622395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.687298 (+1064903us) service_pool.cc:224] Handling call
0709 13:01:27.687672 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.733663 (+ 45991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.526016 (+792353us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b54001b8854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:28.526206 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74063232}
I20260709 13:01:28.536845  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.542819  8892 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:28.552393  8740 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:28.555249  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.563045  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.524219  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:28.585193  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.625676  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.553503  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:28.628304  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 85866.8us
I20260709 13:01:28.629101  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.541733 12783 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.517750 13002 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.642467 12783 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.645152  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.645510  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.646009  9440 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:28.646311  8890 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:28.647131  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.647831  8890 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:28.649427  9444 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:28.649444 12933 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.645179  9449 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:28.650584 13027 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.648568  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.651309  8890 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.504415 12850 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:28.520426 12987 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:28.756520  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.770947  8890 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:28.803360  8893 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.712121  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.804116  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.784209  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.805249  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.756976  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.783816  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.787204  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:28.710215  8892 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:28.818503  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.819469  8892 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.835239  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.835562  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:28.806321  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.838608  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.831184  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.658965 12838 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:28.841832  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.843696  8748 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:28.785501  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.850255  9446 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:28.843091  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.852268  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.837033  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:28.864989 13047 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.659s	user 0.000s	sys 0.010s
I20260709 13:01:28.867656  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2381) took 2275 ms. Trace:
I20260709 13:01:28.868163  8819 rpcz_store.cc:276] 0709 13:01:26.591881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.671783 (+1079902us) service_pool.cc:224] Handling call
0709 13:01:27.682313 (+ 10530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.688411 (+  6098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.688454 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.702831 (+ 14377us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.048494 (+345663us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:28.048549 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.048643 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.864220 (+815577us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:28.867271 (+  3051us) raft_consensus.cc:1685] finished
0709 13:01:28.867299 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.867579 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6107,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5231,"spinlock_wait_cycles":2923904,"thread_start_us":262779,"threads_started":1,"wal-append.queue_time_us":365373}
I20260709 13:01:28.874388  9447 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:28.499265 13036 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:28.793926 13058 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
I20260709 13:01:28.865046  8892 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:28.881394  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.882834  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:28.885980  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54928.6us
W20260709 13:01:28.892944  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 105856us
W20260709 13:01:28.885993  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49051.2us
I20260709 13:01:28.893957  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:28.894290  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:28.886770  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.895864  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.770920 12905 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.724483 12433 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:28.890655  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.885020  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.892699  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:28.903820  8892 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:28.903753  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.904588  8748 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:28.897221  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.902920  8819 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:28.891359  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.913043  9445 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:28.914290  9449 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:28.916574  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.916729  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.917277  9440 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.918560  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:28.921911  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 686) took 1577 ms (client timeout 1780 ms). Trace:
W20260709 13:01:28.922338  8824 rpcz_store.cc:269] 0709 13:01:27.344985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.396138 (+ 51153us) service_pool.cc:224] Handling call
0709 13:01:28.921164 (+1525026us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54003e3fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:28.921274 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512800640}
I20260709 13:01:28.925452  9446 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:28.927765  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2318) took 1070 ms. Trace:
I20260709 13:01:28.928208  8892 rpcz_store.cc:276] 0709 13:01:27.857071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.892492 (+1035421us) service_pool.cc:224] Handling call
0709 13:01:28.927699 (+ 35207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.932304  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.933180  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.933859  8748 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:28.934690  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.925949  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.937032  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:28.937713  9445 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:28.937819  9446 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.942512  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.944343  8892 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:28.944648  8816 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.614s	user 0.003s	sys 0.007s
I20260709 13:01:28.948163  8816 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:28.949817  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.951244  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.947448  8893 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:28.889150 13015 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.685179 12850 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:28.890235 12987 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.889997 12751 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:28.961745  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:28.900462 12997 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:28.965698  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.969192  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 667) took 3608 ms. Trace:
I20260709 13:01:29.001225  8891 rpcz_store.cc:276] 0709 13:01:25.361209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.627911 (+1266702us) service_pool.cc:224] Handling call
0709 13:01:26.628186 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.651280 (+1023094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.651310 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.651354 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:27.651386 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.968876 (+1317490us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:28.968926 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.969115 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353561472}
I20260709 13:01:29.002259  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:28.883754 12889 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:28.523672 13026 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:28.920230 13031 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:28.902479 12237 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:28.975553  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2312) took 1092 ms. Trace:
I20260709 13:01:28.962592  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:28.994387  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:28.905001 12908 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:29.005748  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:28.969136 12975 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.888967 12838 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.065865  8893 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:28.982014  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:29.066989  8748 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:29.013754  9446 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:28.920670 12905 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:28.972251 13019 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.000s
I20260709 13:01:29.056540  8892 rpcz_store.cc:276] 0709 13:01:27.853111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.928963 (+1075852us) service_pool.cc:224] Handling call
0709 13:01:28.944151 (+ 15188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.944813 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.945032 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.995878  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:29.078739  8741 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.025852 12847 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:28.995313 12980 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.000s
W20260709 13:01:29.066269 13058 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:01:29.069190  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:29.033793  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:28.931212 12433 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:29.057680 13037 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.829s	user 0.009s	sys 0.000s
I20260709 13:01:29.063438  8816 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:29.054522 12997 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:29.101615  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:29.092723  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:29.105772  9445 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:29.102867  8819 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:29.108201  8890 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:29.006857 12751 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:29.000348 12987 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.082307 12237 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:29.084316 12847 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:29.134084 12433 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:29.133519 12847 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.134948 12847 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.096894 12742 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:29.120972 12979 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.000s
I20260709 13:01:29.065817 12908 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:29.138446 13032 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.765s	user 0.007s	sys 0.002s
I20260709 13:01:29.068658 12975 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:29.174360  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 247208us
I20260709 13:01:29.134975 12433 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.124272 12237 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.101521 12980 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:29.177136  8789 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.730s (SENT)
I20260709 13:01:29.060345  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:29.087069 13019 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:29.157997 12924 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.929s	user 0.002s	sys 0.008s
W20260709 13:01:29.178903  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 284791us
W20260709 13:01:29.178086  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 284187us
I20260709 13:01:29.176031  9440 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:29.110108 13037 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:29.203867  8743 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:29.206018  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2321) took 1347 ms. Trace:
I20260709 13:01:29.209144  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:29.211746  8892 rpcz_store.cc:276] 0709 13:01:27.858285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.078731 (+1220446us) service_pool.cc:224] Handling call
0709 13:01:29.106814 (+ 28083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.107100 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:29.107392 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.205739 (+ 98347us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.205784 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:29.205821 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.205876 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.205969 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":547,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":13463,"thread_start_us":98022,"threads_started":1}
I20260709 13:01:29.189087 12237 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:29.218027  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2062) took 4304 ms. Trace:
I20260709 13:01:29.211413  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2384) took 2595 ms. Trace:
W20260709 13:01:29.216121  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 155912us
I20260709 13:01:29.203382  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:29.218397  8889 rpcz_store.cc:276] 0709 13:01:24.913921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.365473 (+1451552us) service_pool.cc:224] Handling call
0709 13:01:26.943941 (+578468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.305191 (+361250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.217848 (+1912657us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b5400247f50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:29.217975 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275462912}
W20260709 13:01:29.219794  8789 leader_election.cc:336] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.768s (SENT)
I20260709 13:01:29.230381  8822 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:29.165052 12979 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:29.234972  8789 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.913s (SENT)
I20260709 13:01:29.236428  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:29.213721 12988 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:29.216912 12622 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.247994 12905 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:29.252476 12783 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.000s
W20260709 13:01:29.231621 13039 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.912s	user 0.013s	sys 0.000s
W20260709 13:01:29.248086 12314 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.786s	user 0.007s	sys 0.000s
I20260709 13:01:29.220788 12742 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.190140 12433 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:29.178565 13032 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:29.492810  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 254660us
I20260709 13:01:29.213615 12924 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.228593 12637 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.250s
W20260709 13:01:29.480526 13081 net_util.cc:396] Time spent resolve address for 127.7.117.129: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:01:29.537572  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2324) took 1678 ms. Trace:
I20260709 13:01:29.538057  8892 rpcz_store.cc:276] 0709 13:01:27.859178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.213199 (+1354021us) service_pool.cc:224] Handling call
0709 13:01:29.214331 (+  1132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.214535 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:29.248939 (+ 34404us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.537035 (+288096us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.537190 (+   155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.537234 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:29.537305 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.537386 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.537510 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18796,"prepare.queue_time_us":34865,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"thread_start_us":286989,"threads_started":1}
W20260709 13:01:29.543689  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 321789us
I20260709 13:01:29.544755  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:29.219915 12997 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:29.562179  8745 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:29.560089  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 351148us
I20260709 13:01:29.649346  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:29.533464  8817 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:29.530933 13038 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.023s	user 0.010s	sys 0.000s
W20260709 13:01:29.507417 13066 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.544s	user 0.000s	sys 0.001s
I20260709 13:01:29.650287  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:29.652647  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:29.578867  8892 pending_rounds.cc:85] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 5
I20260709 13:01:29.653803  8892 pending_rounds.cc:107] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:01:29.654068  8892 raft_consensus.cc:2889] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:29.505271 13039 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:29.543437 13083 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:29.510213 13036 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:29.506109 12314 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:29.606051 13077 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:29.606649 13082 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:29.611090  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66427us
I20260709 13:01:29.481429 12783 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.606688 13088 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:29.633096 13085 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 13:01:29.652758 12980 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:29.546178 13084 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:01:29.635340 12987 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:29.728550  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:29.730361  8712 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.887s (SENT) [suppressed 44 similar messages]
I20260709 13:01:29.656186 13037 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:29.572606  8820 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:29.655046 13094 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:29.753126  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.005s	sys 0.021s
W20260709 13:01:29.836258  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.005s	sys 0.021s
I20260709 13:01:29.836704  8740 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:29.837054  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 943550us
I20260709 13:01:29.707361 13083 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.816870  8742 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:29.708959 12975 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:29.838141  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:29.815230 13026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.025s
I20260709 13:01:29.741084 12732 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:29.740895 13079 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:29.663405 13038 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:29.836104 13094 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:29.857523 13026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.003s	sys 0.026s
I20260709 13:01:28.506255  8821 rpcz_store.cc:276] 0709 13:01:27.344616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.333069 (+988453us) service_pool.cc:224] Handling call
0709 13:01:28.333389 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.338256 (+  4867us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:28.395397 (+ 57141us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.487017 (+ 91620us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.492805 (+  5788us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.493236 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.493432 (+   196us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:28.493463 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.493526 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.504082 (+ 10556us) raft_consensus.cc:1685] finished
0709 13:01:28.504137 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.505768 (+  1631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27075,"prepare.queue_time_us":75743,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":42729,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":12828,"spinlock_wait_cycles":5574272,"thread_start_us":42649,"threads_started":2,"wal-append.queue_time_us":86106}
I20260709 13:01:29.708395 13039 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:29.788323 12908 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:29.839509  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111100us
I20260709 13:01:29.744231 13088 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:29.919143 12944 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.693s	user 0.001s	sys 0.011s
I20260709 13:01:29.845674 13083 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.741189 12751 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:29.905997  8741 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.000s
I20260709 13:01:29.946199  8741 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:29.947829  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2066) took 1023 ms. Trace:
I20260709 13:01:29.861940 12936 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:29.942538  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:29.877317 13032 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:29.949931  8741 rpcz_store.cc:276] 0709 13:01:28.924192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.932747 (+  8555us) service_pool.cc:224] Handling call
0709 13:01:29.947763 (+1015016us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:29.954609  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.045s	sys 0.062s
I20260709 13:01:29.872208 13019 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:29.920293 12732 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:29.971124  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.045s	sys 0.062s
W20260709 13:01:29.977066  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 742418us
I20260709 13:01:29.975294  8815 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:29.944542 13031 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.930042 13090 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.974905  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:29.981073  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2067) took 1044 ms. Trace:
I20260709 13:01:29.981421  8741 rpcz_store.cc:276] 0709 13:01:28.936413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.974630 (+1038217us) service_pool.cc:224] Handling call
0709 13:01:29.981014 (+  6384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.863281 12979 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:29.927275 12433 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:29.947649 13088 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:29.928377 13094 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:29.977847  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:29.949867 12944 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.949646 12924 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:29.998055 13091 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:01:29.995860 12751 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.985183  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.967224 12936 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:29.971668  8889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.011s
I20260709 13:01:29.988004  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2057) took 1651 ms. Trace:
W20260709 13:01:30.006696  8889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.006s	sys 0.011s
I20260709 13:01:30.008718  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2326) took 2149 ms. Trace:
I20260709 13:01:30.005437  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:30.005674  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:29.996363  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.009754  8898 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.011236 12987 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:30.010809 12732 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.011816 12987 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:30.012866 12783 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:30.009114  8889 rpcz_store.cc:276] 0709 13:01:27.859729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.219332 (+1359603us) service_pool.cc:224] Handling call
0709 13:01:29.234785 (+ 15453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.234973 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:29.253546 (+ 18573us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.008410 (+754864us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.008457 (+    47us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:30.008496 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.008555 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.008664 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19431,"prepare.queue_time_us":37134,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":753325,"threads_started":1}
W20260709 13:01:30.011615 12759 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.779s	user 0.006s	sys 0.006s
W20260709 13:01:29.992561 13074 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:01:30.001900 12751 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:30.006757  8744 rpcz_store.cc:276] 0709 13:01:28.337005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.337197 (+   192us) service_pool.cc:224] Handling call
0709 13:01:28.337539 (+   342us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:28.366534 (+ 28995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.366567 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:28.392704 (+ 26137us) log.cc:844] Serialized 78 byte log entry
0709 13:01:28.500246 (+107542us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:28.500299 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.500372 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.987699 (+1487327us) raft_consensus.cc:1685] finished
0709 13:01:29.987728 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.987930 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6160,"raft.queue_time_us":5359,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":24201,"thread_start_us":91832,"threads_started":1,"wal-append.queue_time_us":434631}
I20260709 13:01:30.017865  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:30.021544  8824 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
W20260709 13:01:30.010874 12933 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.361s	user 0.010s	sys 0.001s
I20260709 13:01:30.024181  8744 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:30.020740  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.026639  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:29.991060 13099 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:30.005189 13087 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.031239  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1761) took 1637 ms. Trace:
I20260709 13:01:30.031205  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.031811  8744 rpcz_store.cc:276] 0709 13:01:28.393445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.021302 (+1627857us) service_pool.cc:224] Handling call
0709 13:01:30.021627 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.026043 (+  4416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.031175 (+  5132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.034096  8744 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:30.034853  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1769) took 1635 ms. Trace:
I20260709 13:01:30.035302  8744 rpcz_store.cc:276] 0709 13:01:28.399818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.033542 (+1633724us) service_pool.cc:224] Handling call
0709 13:01:30.033800 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.034543 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.034787 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.035625  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:30.039708  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:30.065395  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 194347us
I20260709 13:01:30.037693  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1772) took 1619 ms. Trace:
I20260709 13:01:30.017467 13092 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.076040  8744 rpcz_store.cc:276] 0709 13:01:28.418704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.036583 (+1617879us) service_pool.cc:224] Handling call
0709 13:01:30.036994 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.037232 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.037269 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.037351 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:30.037398 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.037468 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.037627 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:30.115423  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 385263us
I20260709 13:01:29.991700 12975 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:30.017900 12908 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:30.026115 12924 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:30.182335 13075 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:30.182960 13075 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:30.191537 13073 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.072s	user 0.001s	sys 0.005s
W20260709 13:01:30.192418 12994 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.749s	user 0.011s	sys 0.000s
W20260709 13:01:30.196030 13097 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:01:30.197782 12908 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.069977  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.121692 13080 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:30.127985 13095 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:01:30.129797 13027 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.479s	user 0.008s	sys 0.004s
I20260709 13:01:30.130667 12987 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:30.134464 13016 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.742s	user 0.007s	sys 0.004s
I20260709 13:01:30.134833 12936 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.135931  8898 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:30.137004 13083 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.142969 12751 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.145079 12783 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:30.145838 13086 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:01:30.146941 13038 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:30.148265 13093 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:01:30.149592 12237 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.918s	user 0.006s	sys 0.006s
I20260709 13:01:30.025625 12933 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.152383  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 674) took 3890 ms. Trace:
I20260709 13:01:30.153928 13088 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:30.160087  8713 connection.cc:256] RPC call timeout handler was delayed by -1.1216609329999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 13:01:30.163569  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.164538  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.168000  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.019955 12759 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.177417  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2390) took 1914 ms. Trace:
W20260709 13:01:30.214389  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.032555 13082 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.033449 13037 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:30.581251  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1767) took 2181 ms (client timeout 1736 ms). Trace:
W20260709 13:01:30.588660  8742 rpcz_store.cc:269] 0709 13:01:28.399447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.501633 (+102186us) service_pool.cc:224] Handling call
0709 13:01:30.581013 (+2079380us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540016e4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.581174 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758157568}
I20260709 13:01:30.582484  8889 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:01:30.259891  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.589807  8822 rpcz_store.cc:276] 0709 13:01:28.262624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.539318 (+276694us) service_pool.cc:224] Handling call
0709 13:01:28.539696 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.232560 (+692864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.177062 (+944502us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54003ec1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.177327 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588702464}
I20260709 13:01:30.567181  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:30.591639  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 521973us
I20260709 13:01:30.592173  8748 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.581988  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 680) took 2051 ms. Trace:
I20260709 13:01:30.592762  8890 rpcz_store.cc:276] 0709 13:01:28.530649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.838333 (+307684us) service_pool.cc:224] Handling call
0709 13:01:30.581819 (+1743486us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54001806d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.581932 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618423040}
I20260709 13:01:30.592993  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:30.594238  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 652054us
I20260709 13:01:30.595379  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.596943  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:30.304036  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 243733us
I20260709 13:01:30.490743  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2316) took 2636 ms. Trace:
I20260709 13:01:30.598708  9445 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:30.598942  8893 rpcz_store.cc:276] 0709 13:01:27.854346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.952803 (+1098457us) service_pool.cc:224] Handling call
0709 13:01:28.953177 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.066439 (+113262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.490423 (+1423984us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540015d954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.490671 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258533120}
I20260709 13:01:29.220036  8823 rpcz_store.cc:276] 0709 13:01:26.616367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.682787 (+1066420us) service_pool.cc:224] Handling call
0709 13:01:27.683062 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.683277 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:27.702432 (+ 19155us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.685330 (+982898us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b1800100720. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:01:29.210827 (+525497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.210882 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:29.211009 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.211242 (+   233us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.211351 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1301,"prepare.queue_time_us":18896,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":127728,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5510,"spinlock_wait_cycles":313748480,"thread_start_us":465211,"threads_started":2,"wal-append.queue_time_us":1340940}
I20260709 13:01:30.599884  9448 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:30.552875  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.421452  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 853) took 1963 ms. Trace:
I20260709 13:01:30.601472  9444 rpcz_store.cc:276] 0709 13:01:28.458443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.458565 (+   122us) service_pool.cc:224] Handling call
0709 13:01:28.474365 (+ 15800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.649838 (+175473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.421109 (+1771271us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400634554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.421379 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402113408}
W20260709 13:01:30.489415  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1749) took 2101 ms (client timeout 1505 ms). Trace:
W20260709 13:01:30.602921  8743 rpcz_store.cc:269] 0709 13:01:28.387589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.387704 (+   115us) service_pool.cc:224] Handling call
0709 13:01:30.489053 (+2101349us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b540013bfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.489274 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861630848}
I20260709 13:01:30.115280 12742 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.588878  8896 rpcz_store.cc:276] 0709 13:01:26.261843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.364635 (+1102792us) service_pool.cc:224] Handling call
0709 13:01:27.777442 (+412807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.777672 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:28.057249 (+279577us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.219273 (+162024us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.234095 (+ 14822us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.236795 (+  2700us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:28.237706 (+   911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.237830 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.238719 (+1000889us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b18002a9e80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:30.152195 (+913476us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400248950. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.152326 (+   131us) inbound_call.cc:177] Queueing success response
0709 13:01:30.171386 (+ 19060us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18002a9e80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
Metrics: {"mutex_wait_us":4605,"prepare.queue_time_us":261813,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2238,"raft.queue_time_us":15753,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":11733,"spinlock_wait_cycles":974307328,"thread_start_us":423060,"threads_started":2,"wal-append.queue_time_us":228993,"wal-append.run_cpu_time_us":2535,"wal-append.run_wall_time_us":1227645}
W20260709 13:01:30.204811 13074 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:01:30.624558  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:30.618113  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:30.617012  8864 connection.cc:256] RPC call timeout handler was delayed by -1.2381854249999833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:01:30.609655  8890 raft_consensus.cc:1217] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-12.7]   Dedup: 8.5->[10.6-12.7]
I20260709 13:01:30.632342  8743 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.632723  8890 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: [8.5-12.7]
I20260709 13:01:30.633656  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2099) took 2402 ms. Trace:
I20260709 13:01:30.634083  8890 rpcz_store.cc:276] 0709 13:01:28.231287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.608987 (+2377700us) service_pool.cc:224] Handling call
0709 13:01:30.609439 (+   452us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:30.633350 (+ 23911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.633591 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.632630  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.629355  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:30.325554 13033 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.826s	user 0.013s	sys 0.000s
I20260709 13:01:30.639500 13088 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:30.639303 13033 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.609187 12944 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:30.641500 13033 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:30.641361 12933 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:30.641346 12759 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:30.252717 13027 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:30.645417 13094 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.000s
I20260709 13:01:30.645572 13027 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.645886 13094 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.648366 13094 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:30.648612  8748 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:30.648916 13094 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.589550 12783 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:30.629825 13015 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.634s	user 0.013s	sys 0.000s
I20260709 13:01:30.609964 12807 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:30.527200 12892 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:30.637925  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:30.520515 13039 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.192998 12994 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.295166 12314 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.597630 12947 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:30.690052 12947 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.690587  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:30.692075  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.692293 12947 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:30.638491  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.692891  9440 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:30.693007  8893 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.624819 13016 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.696696 13016 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.610410 12237 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.666770 12936 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.699147  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2093) took 2670 ms. Trace:
I20260709 13:01:30.699590 12237 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:30.635739  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.699728  8889 rpcz_store.cc:276] 0709 13:01:28.028996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.020591 (+1991595us) service_pool.cc:224] Handling call
0709 13:01:30.581986 (+561395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.590038 (+  8052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.698956 (+108918us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54001806d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.699091 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262532480}
I20260709 13:01:30.700662  8896 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.701102  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.701754  8889 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:30.702636  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.702842  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:30.703130  8889 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.703678  9448 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:30.705901  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 691) took 2888 ms. Trace:
I20260709 13:01:30.706472  8820 rpcz_store.cc:276] 0709 13:01:27.817070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.381023 (+563953us) service_pool.cc:224] Handling call
0709 13:01:28.385884 (+  4861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.837802 (+1451918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.705654 (+867852us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540040a454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.705819 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729936768}
I20260709 13:01:30.707579  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:30.707768  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2319) took 2850 ms. Trace:
I20260709 13:01:30.650568 13083 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:30.708285  8891 rpcz_store.cc:276] 0709 13:01:27.857651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.003054 (+1145403us) service_pool.cc:224] Handling call
0709 13:01:29.003435 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.263058 (+259623us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:29.263106 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:29.478443 (+215337us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.938392 (+459949us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:29.938452 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.938561 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.707526 (+768965us) raft_consensus.cc:1685] finished
0709 13:01:30.707551 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.707703 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130844,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":637317,"threads_started":2,"wal-append.queue_time_us":469934}
I20260709 13:01:30.650553  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:30.710572  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.199064 13075 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:30.712186  8895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2328) took 2834 ms (client timeout 1887 ms). Trace:
W20260709 13:01:30.712597  8895 rpcz_store.cc:269] 0709 13:01:27.881944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.235140 (+353196us) service_pool.cc:224] Handling call
0709 13:01:30.712000 (+2476860us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000aa5d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.712091 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435888384}
I20260709 13:01:30.712968  8889 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:30.713701  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:30.713943 13072 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.626s	user 0.000s	sys 0.007s
I20260709 13:01:30.714435  8895 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.672822 13015 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.715808  9446 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:30.651093 13002 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.231693 13031 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:30.654959  8743 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:30.722095  8743 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:30.722564  9441 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:30.704418  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:30.656021  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1743) took 1771 ms (client timeout 1888 ms). Trace:
W20260709 13:01:30.723322  8817 rpcz_store.cc:269] 0709 13:01:28.884689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.884813 (+   124us) service_pool.cc:224] Handling call
0709 13:01:30.655868 (+1771055us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540041ffd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.655968 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324129152}
I20260709 13:01:30.658744 12838 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:30.659054 13101 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:01:30.660718  8749 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:30.715773  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.205852 12850 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.281838 13079 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:30.672400  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 817) took 2198 ms. Trace:
I20260709 13:01:30.730460  9447 rpcz_store.cc:276] 0709 13:01:28.469130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.543618 (+ 74488us) service_pool.cc:224] Handling call
0709 13:01:28.850855 (+307237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.874889 (+ 24034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.667514 (+1792625us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54005f23d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.667669 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706183936}
I20260709 13:01:30.609879 13023 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:30.635002 13020 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:30.625131 12742 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:28.643424  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:30.683663  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1747) took 1793 ms. Trace:
I20260709 13:01:30.686355 12807 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:30.666505 12944 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.689096  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 823) took 1907 ms. Trace:
I20260709 13:01:30.735718  8816 rpcz_store.cc:276] 0709 13:01:28.890134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.103729 (+213595us) service_pool.cc:224] Handling call
0709 13:01:29.104084 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.176989 (+ 72905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.177019 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.177059 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:29.177093 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.683360 (+1506267us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b540001f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.683414 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.683591 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167305600}
I20260709 13:01:30.697567  9443 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:30.608891 12682 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.705778 13104 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:30.655148  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2061) took 2307 ms (client timeout 1981 ms). Trace:
I20260709 13:01:30.725155  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1750) took 2337 ms. Trace:
I20260709 13:01:30.198597 12908 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:30.729758 12905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.019s	sys 0.033s
I20260709 13:01:30.728078  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.736585 12742 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.729933  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:30.747958  8820 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.735872  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.750669  8889 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:30.750780  8898 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:30.731868  8896 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:30.125738 13092 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.669144 13094 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:30.724025  9448 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:30.737694  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:30.755925  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:30.736898  9449 rpcz_store.cc:276] 0709 13:01:28.781611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.836564 (+ 54953us) service_pool.cc:224] Handling call
0709 13:01:28.914042 (+ 77478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.923794 (+  9752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.688850 (+1765056us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b54006dd650. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.689024 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177391488}
W20260709 13:01:30.738472  8745 rpcz_store.cc:269] 0709 13:01:28.347705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.347819 (+   114us) service_pool.cc:224] Handling call
0709 13:01:30.654942 (+2307123us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54004650d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.655079 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712654080}
I20260709 13:01:30.738854  8747 rpcz_store.cc:276] 0709 13:01:28.387875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.388010 (+   135us) service_pool.cc:224] Handling call
0709 13:01:28.389136 (+  1126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.414634 (+ 25498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.724951 (+2310317us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b540040c9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.725104 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37200128}
I20260709 13:01:30.736205 12807 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:30.742368  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2331) took 2859 ms. Trace:
I20260709 13:01:30.746915  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.757463  8892 rpcz_store.cc:276] 0709 13:01:27.882757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.578090 (+1695333us) service_pool.cc:224] Handling call
0709 13:01:29.578593 (+   503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:29.711417 (+132824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.711447 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:29.742348 (+ 30901us) log.cc:844] Serialized 52 byte log entry
0709 13:01:30.018539 (+276191us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:30.018589 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.018654 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.742126 (+723472us) raft_consensus.cc:1685] finished
0709 13:01:30.742146 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.742312 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30266,"raft.queue_time_us":175361,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":61971,"thread_start_us":156665,"threads_started":1,"wal-append.queue_time_us":485817}
I20260709 13:01:30.759977  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.760625  8892 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:30.746690  8893 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:30.760938  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:30.763000  8821 raft_consensus.cc:2410] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:30.763607  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.764417  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:30.764441  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:30.765462  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.008s	sys 0.013s
I20260709 13:01:30.765610  8817 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.767956  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.752660  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.803683  8896 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:30.738490 13020 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:30.745302 12905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.019s	sys 0.033s
I20260709 13:01:30.756227  8816 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:30.694133  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:30.751471  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.837399  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:30.775990 13104 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:30.730043 12732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.006s	sys 0.013s
I20260709 13:01:30.747296 13031 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.777714  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:30.749974 12987 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:30.779565  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:30.780094  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.780428  8891 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.769366 12682 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:30.791128  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1762) took 2392 ms (client timeout 1888 ms). Trace:
W20260709 13:01:30.868013  8749 rpcz_store.cc:269] 0709 13:01:28.398571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.515533 (+116962us) service_pool.cc:224] Handling call
0709 13:01:30.790958 (+2275425us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400021cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.791058 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4840009600}
I20260709 13:01:30.791760  8744 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:30.765786  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.008s	sys 0.013s
W20260709 13:01:30.869439  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 793243us
I20260709 13:01:30.812896  8820 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:30.819850  8817 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:30.830214  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2063) took 1906 ms (client timeout 2415 ms). Trace:
W20260709 13:01:30.870990  8740 rpcz_store.cc:269] 0709 13:01:28.923396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.994015 (+ 70619us) service_pool.cc:224] Handling call
0709 13:01:30.829966 (+1835951us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540013b5d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.830140 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790949632}
I20260709 13:01:30.843032  8816 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:30.735718 13023 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.838091  8889 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:30.838920  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1736) took 4251 ms. Trace:
W20260709 13:01:30.834944  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.016s	sys 0.049s
W20260709 13:01:30.876741  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.016s	sys 0.049s
W20260709 13:01:30.843343  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 857) took 2330 ms (client timeout 2207 ms). Trace:
I20260709 13:01:30.838886  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 687) took 3493 ms. Trace:
I20260709 13:01:30.870077  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.878378  8815 rpcz_store.cc:276] 0709 13:01:27.345363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.334102 (+988739us) service_pool.cc:224] Handling call
0709 13:01:28.334353 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.978358 (+1644005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.838645 (+860287us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540004d8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.838831 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3721561472}
I20260709 13:01:30.858291  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:30.866767  8819 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:30.879886  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.879902  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:30.859309 12732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.006s	sys 0.014s
I20260709 13:01:30.886116  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:30.877497  9443 rpcz_store.cc:269] 0709 13:01:28.459734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.459849 (+   115us) service_pool.cc:224] Handling call
0709 13:01:30.789915 (+2330066us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54005b09d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.790062 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5034039552}
I20260709 13:01:30.885260  8895 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
W20260709 13:01:30.890865  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61730.1us
I20260709 13:01:30.835793 13102 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:30.894636  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:30.894870  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.895665  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:30.907720  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.908313  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.909103  8893 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.875188  8818 rpcz_store.cc:276] 0709 13:01:26.587588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.622297 (+1034709us) service_pool.cc:224] Handling call
0709 13:01:28.924397 (+1302100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.179234 (+254837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.179262 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:29.179682 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.501521 (+321839us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:29.501581 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.501780 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.643057 (+1141277us) raft_consensus.cc:1685] finished
0709 13:01:30.643078 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.790424 (+147346us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003e3fd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.838866 (+ 48442us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74233,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":22970,"spinlock_wait_cycles":2941393280,"thread_start_us":539593,"threads_started":2,"wal-append.queue_time_us":433212}
I20260709 13:01:30.910593  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.912817  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:30.914829  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.919432  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:30.854367 12237 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:30.853881 13083 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:01:30.922241  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 771) took 2042 ms. Trace:
I20260709 13:01:30.923964  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:30.924691  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.930806  9446 rpcz_store.cc:276] 0709 13:01:28.880180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.068182 (+188002us) service_pool.cc:224] Handling call
0709 13:01:29.068578 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.716198 (+1647620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.922029 (+205831us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54001955d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.922192 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3789608192}
I20260709 13:01:30.782444 13079 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:30.926192  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2064) took 2002 ms. Trace:
I20260709 13:01:30.926811 13023 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.919574 13082 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:30.927219  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 860) took 2466 ms. Trace:
W20260709 13:01:30.928784 13080 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.000s
I20260709 13:01:30.865806 12905 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:01:30.931988  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:30.940716  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:30.907037 13037 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:30.932492  8744 rpcz_store.cc:276] 0709 13:01:28.923641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.124317 (+1200676us) service_pool.cc:224] Handling call
0709 13:01:30.791579 (+667262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.869325 (+ 77746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.926007 (+ 56682us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b5400021cd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:30.926136 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534400768}
I20260709 13:01:30.947925  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:30.932018  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:30.894047 12889 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.894213 12759 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:30.885844 13039 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:30.892298 13038 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:30.934504  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 306767us
I20260709 13:01:30.935488  9441 rpcz_store.cc:276] 0709 13:01:28.460610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.520251 (+ 59641us) service_pool.cc:224] Handling call
0709 13:01:28.652741 (+132490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.724875 (+2072134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.927002 (+202127us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540059f6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:30.927166 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4762041600}
I20260709 13:01:30.783962 12807 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:30.983713  8741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.010s	sys 0.005s
W20260709 13:01:30.936264  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 318399us
I20260709 13:01:30.897763 13102 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:30.889916 12314 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:30.941412  9446 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.956197 12732 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:30.941705  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
W20260709 13:01:30.961309  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.936731 12975 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:30.948324 13030 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:30.984174  8741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.010s	sys 0.005s
I20260709 13:01:30.947283 12994 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:30.948020 13080 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.877646 12987 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:30.984288  8894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.003s	sys 0.011s
I20260709 13:01:30.984005 13038 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:31.055514  8894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.003s	sys 0.015s
I20260709 13:01:30.963271 13103 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.958292 13023 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.017762  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:31.010430 13102 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.956974 12889 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.034878  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:30.962827 13026 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:31.048518  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2399) took 1032 ms. Trace:
I20260709 13:01:30.981570 13082 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:30.957939 12237 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:30.982014 13037 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.012113  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:31.017860  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 426842us
I20260709 13:01:31.135609  8817 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:01:31.100567 13005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.936s	user 0.010s	sys 0.040s
I20260709 13:01:31.074183  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:31.006011 13039 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:31.067622  8893 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:31.141896  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:31.143018  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:30.999274 12944 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:31.103116  8824 rpcz_store.cc:276] 0709 13:01:30.016139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.031429 (+ 15290us) service_pool.cc:224] Handling call
0709 13:01:30.033817 (+  2388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.033996 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:30.162719 (+128723us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.048012 (+885293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.048083 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:31.048162 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.048261 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.048438 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128359,"prepare.queue_time_us":131089,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":70736,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":31885,"thread_start_us":884761,"threads_started":2,"wal-append.queue_time_us":490667}
I20260709 13:01:31.024936 12975 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:31.151914  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 344985us
I20260709 13:01:31.016220 12732 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:31.024561 12908 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:31.138787 13005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.018s	user 0.010s	sys 0.040s
I20260709 13:01:31.061228  9446 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
W20260709 13:01:31.075310  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57646.9us
I20260709 13:01:31.075250 13038 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.139283  9441 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:31.136723  8787 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.092s (SENT)
I20260709 13:01:31.131959 13082 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:31.146138  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 858) took 2686 ms. Trace:
W20260709 13:01:31.163033  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 335764us
I20260709 13:01:31.030102 12314 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.045347 13109 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 13:01:31.133905 13103 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.161718  9442 rpcz_store.cc:276] 0709 13:01:28.460097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.460204 (+   107us) service_pool.cc:224] Handling call
0709 13:01:30.709774 (+2249570us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:30.709916 (+   142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.726486 (+ 16570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.726530 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.729788 (+  3258us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.063263 (+333475us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:31.063320 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.063417 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.145812 (+ 82395us) raft_consensus.cc:1685] finished
0709 13:01:31.145833 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.146079 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16416,"raft.queue_time_us":16310,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4400,"spinlock_wait_cycles":2557184,"thread_start_us":320072,"threads_started":1,"wal-append.queue_time_us":401880}
I20260709 13:01:31.158617  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:31.180948  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.100554  8891 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:31.101056 13030 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:31.044921 12759 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:31.164553  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 255056us
I20260709 13:01:30.940464 13020 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:31.108438 12889 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.134877 13026 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:31.155711 12975 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.102746 13023 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:31.187393  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 279840us
I20260709 13:01:31.169162 13082 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.196373  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:31.197872  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.168175 13094 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:31.205019  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.224004  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.183600 12908 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.219270 13114 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:31.220775 13113 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:01:31.222745  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 55566.4us
I20260709 13:01:31.233069  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:31.234385  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:31.251370  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.251962  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:31.225363 13110 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 13:01:31.259361  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 247329us
I20260709 13:01:31.260906  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:31.260849  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 226146us
I20260709 13:01:31.262017  8817 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:31.262199  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65949.2us
I20260709 13:01:31.271675  8863 leader_election.cc:400] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:31.272163  8863 leader_election.cc:403] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:31.184588 13103 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.232975 12988 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.235706 13080 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:31.290606  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2334) took 3250 ms. Trace:
I20260709 13:01:31.290992  8894 rpcz_store.cc:276] 0709 13:01:28.040403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.334426 (+2294023us) service_pool.cc:224] Handling call
0709 13:01:30.408207 (+ 73781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.408487 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:31.056282 (+647795us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.290260 (+233978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.290315 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:31.290357 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.290428 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.290549 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24790,"prepare.queue_time_us":300500,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":856006,"threads_started":2}
I20260709 13:01:31.286592  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.292701  8824 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.257905 13094 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:31.297153  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.312153  9446 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:31.299248  8817 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:31.299633  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:31.314471  9440 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.306221 12988 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.278635 12997 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:31.283597 13111 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 13:01:31.394541  9440 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:31.316725  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.323235 13128 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:31.258145 13106 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:01:31.326982 13103 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:31.328691 12988 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:31.330468  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:31.402364 13103 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:31.403247  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:31.403946  8894 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:31.331385  8816 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.281759 12759 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.337977  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:31.342512 13023 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:31.344688 12936 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.003s
I20260709 13:01:31.287683 13030 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:31.349793 13115 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:01:31.353518 12944 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.476056  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:31.353729 13118 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:01:31.354401 13123 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:01:31.485833  8894 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:31.486578  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2103) took 3146 ms. Trace:
I20260709 13:01:31.486937  8894 rpcz_store.cc:276] 0709 13:01:28.340472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.445388 (+3104916us) service_pool.cc:224] Handling call
0709 13:01:31.485631 (+ 40243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.486320 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.486520 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.396396  8817 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:31.304977 13088 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:31.342427 12742 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.005s
I20260709 13:01:31.309646 13026 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:31.253425 12732 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.421034 12924 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:01:31.399358 12806 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:31.431450  8824 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:31.477355  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.489027  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.418717 13131 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:31.485481 13027 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:31.471603 13083 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:31.501374 13107 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.568s	user 0.002s	sys 0.000s
I20260709 13:01:31.438755 12751 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:31.482985 13111 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:01:31.526160  8894 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:31.547911  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.553040  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 676) took 3023 ms. Trace:
I20260709 13:01:31.555214  8894 rpcz_store.cc:276] 0709 13:01:28.529724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.487735 (+2958011us) service_pool.cc:224] Handling call
0709 13:01:31.525980 (+ 38245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.552818 (+ 26838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.552990 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.484495  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 221672us
I20260709 13:01:31.557186  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.557976  8824 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:31.442993 13117 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:31.479998  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 186293us
W20260709 13:01:31.517030 13127 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:31.559643  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:31.553390  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.561719  8817 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.540766  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:31.567926  8815 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:31.477355 13023 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.568852  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:31.581867  8815 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:31.518333 12936 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:31.584375 13023 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.584820 12936 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.585227 12936 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:31.585985 13102 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.586583 13023 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:31.586094 12936 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.473737  8816 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:31.541853 12889 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.587805  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:31.588333 13102 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:31.548278 12924 consensus_queue.cc:1243] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:31.546379 13039 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:31.575196  8894 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:31.624564  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 682) took 3093 ms. Trace:
I20260709 13:01:31.624987  8894 rpcz_store.cc:276] 0709 13:01:28.531094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.556220 (+3025126us) service_pool.cc:224] Handling call
0709 13:01:31.574790 (+ 18570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.624285 (+ 49495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.624507 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.576375 13124 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.289s	user 0.000s	sys 0.000s
I20260709 13:01:31.576735 12944 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.576455 13082 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:31.577162 13133 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:01:31.577736 13068 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 2.457s	user 0.005s	sys 0.007s
I20260709 13:01:31.539120 12997 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:01:31.578403  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2335) took 3331 ms. Trace:
I20260709 13:01:31.544100 13132 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:01:31.521903 12742 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:31.524013 12314 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:31.617930  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 354633us
W20260709 13:01:31.621855  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 324273us
W20260709 13:01:31.622490  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65880.4us
W20260709 13:01:31.630228  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77478.3us
I20260709 13:01:31.549878 13083 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:31.590461 13088 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:31.549750 13027 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.622793 12989 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:01:31.550884 12751 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:31.549255 13131 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.635067  8894 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:31.635489  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 698) took 1105 ms. Trace:
I20260709 13:01:32.394095  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 683) took 3862 ms. Trace:
I20260709 13:01:32.394187  8742 rpcz_store.cc:276] 0709 13:01:30.530538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.672750 (+142212us) service_pool.cc:224] Handling call
0709 13:01:30.688431 (+ 15681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.688663 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:31.209652 (+520989us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.633732 (+424080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.633789 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:31.633833 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.633893 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.635430 (+  1537us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":376534,"prepare.queue_time_us":289588,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":257135,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":23482,"thread_start_us":567743,"threads_started":3,"wal-append.queue_time_us":369089}
I20260709 13:01:31.848214  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2404) took 1829 ms. Trace:
I20260709 13:01:31.648423 13160 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.765s
I20260709 13:01:32.398713 12976 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.399776 12976 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:32.400172 12976 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:31.828681  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2098) took 3770 ms. Trace:
I20260709 13:01:32.412472  8898 rpcz_store.cc:276] 0709 13:01:28.058524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.609422 (+2550898us) service_pool.cc:224] Handling call
0709 13:01:30.609752 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.751146 (+141394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.828404 (+1077258us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540046e454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:31.828617 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323963648}
W20260709 13:01:31.675254  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.415534  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.752417  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 476190us
I20260709 13:01:32.416599  8898 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:32.401108 12976 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:31.830248 12975 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.597s	user 0.006s	sys 0.003s
I20260709 13:01:31.649581 12889 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:31.831722 13038 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.000s
I20260709 13:01:32.489707 13038 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.491135 12751 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.492123 13038 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:31.948926  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 685) took 1312 ms. Trace:
I20260709 13:01:32.493834  8890 rpcz_store.cc:276] 0709 13:01:30.636958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.947846 (+1310888us) service_pool.cc:224] Handling call
0709 13:01:31.948248 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.948491 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.948528 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.948584 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:31.948627 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.948695 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.948853 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.494854 12742 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.495327  8890 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:32.496011  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 692) took 1188 ms. Trace:
I20260709 13:01:31.851039  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:31.781090 13117 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:32.496970 13117 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.496973  9447 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.827870  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 684) took 1191 ms. Trace:
I20260709 13:01:32.496327  8890 rpcz_store.cc:276] 0709 13:01:31.307924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.494859 (+1186935us) service_pool.cc:224] Handling call
0709 13:01:32.495155 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.495774 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.495955 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.775234 13157 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.842672  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2410) took 1156 ms. Trace:
I20260709 13:01:31.649015 13159 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.779817  8817 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:32.408586  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:32.492336 13083 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:31.680770  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2397) took 1675 ms. Trace:
I20260709 13:01:32.491710 12751 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:32.411777  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:31.950183 13031 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:32.417389  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 322181us
I20260709 13:01:32.397063  8823 rpcz_store.cc:276] 0709 13:01:30.018755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.625391 (+606636us) service_pool.cc:224] Handling call
0709 13:01:30.625688 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.625896 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:30.643982 (+ 18086us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.847323 (+1203341us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b1800397d80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:01:31.847590 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.847801 (+   211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.847851 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:31.847903 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.847993 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.848143 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40889,"prepare.queue_time_us":1368313,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":81,"raft.queue_time_us":44876,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":55545,"spinlock_wait_cycles":20778880,"thread_start_us":394971,"threads_started":1,"wal-append.queue_time_us":579731}
I20260709 13:01:32.401522  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:32.517482  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:32.517716  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 116344us
I20260709 13:01:32.420552  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.392838  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:32.519517  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 126789us
I20260709 13:01:32.421342  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.389196  8889 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:32.394810  8894 rpcz_store.cc:276] 0709 13:01:28.531333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.625813 (+3094480us) service_pool.cc:224] Handling call
0709 13:01:31.634753 (+  8940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.393772 (+759019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.394035 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.420414  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.437693  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.419116  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 686) took 1592 ms. Trace:
I20260709 13:01:32.409684  8892 rpcz_store.cc:276] 0709 13:01:28.246767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.766052 (+2519285us) service_pool.cc:224] Handling call
0709 13:01:30.766420 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.768264 (+  1844us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:30.777372 (+  9108us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.159927 (+382555us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.160379 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.160523 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:31.160713 (+   190us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.187115 (+ 26402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.187198 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.499862 (+312664us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b180033ca60. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:31.578152 (+ 78290us) raft_consensus.cc:1685] finished
0709 13:01:31.578186 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.578342 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4235,"prepare.queue_time_us":349422,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":73935,"raft.run_cpu_time_us":533,"raft.run_wall_time_us":72909,"spinlock_wait_cycles":3710720,"thread_start_us":269321,"threads_started":1,"wal-append.queue_time_us":610765}
I20260709 13:01:32.442775  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.518587  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:32.481630  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.486557  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:31.802016 12622 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:32.501760  8895 rpcz_store.cc:276] 0709 13:01:30.636644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.826795 (+1190151us) service_pool.cc:224] Handling call
0709 13:01:31.827187 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.827426 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.827463 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.827518 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:31.827560 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.827631 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.827770 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.757035  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 696) took 3526 ms. Trace:
I20260709 13:01:32.168157 13030 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:31.930522 13155 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.507506  9447 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:32.508327  8820 rpcz_store.cc:276] 0709 13:01:30.686326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.707348 (+ 21022us) service_pool.cc:224] Handling call
0709 13:01:31.842442 (+1135094us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54002120d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:31.842596 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91698048}
I20260709 13:01:31.851442  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2338) took 3574 ms. Trace:
I20260709 13:01:32.511052  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 703) took 1279 ms. Trace:
W20260709 13:01:31.638442  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.390674  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.438510 13121 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.512849  8821 rpcz_store.cc:276] 0709 13:01:30.005052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.005408 (+   356us) service_pool.cc:224] Handling call
0709 13:01:31.680501 (+1675093us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:31.680700 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733165440}
I20260709 13:01:32.496534 13154 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.515022  9448 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:32.516919  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.492785 13038 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:01:32.560165  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 871) took 1641 ms. Trace:
I20260709 13:01:32.554796  8746 rpcz_store.cc:276] 0709 13:01:28.230564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.230719 (+   155us) service_pool.cc:224] Handling call
0709 13:01:28.231146 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.237340 (+  6194us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:28.242593 (+  5253us) log.cc:844] Serialized 14 byte log entry
0709 13:01:28.527503 (+284910us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b4800080058. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:01:30.778634 (+2251131us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b18003a6120. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:01:30.784351 (+  5717us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.394595 (+610244us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.395067 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.395216 (+   149us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:31.418393 (+ 23177us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.483796 (+ 65403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.483872 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.756741 (+272869us) raft_consensus.cc:1685] finished
0709 13:01:31.756770 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.756959 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341226,"prepare.queue_time_us":2517665,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":104,"raft.queue_time_us":414718,"raft.run_cpu_time_us":2014,"raft.run_wall_time_us":304345,"spinlock_wait_cycles":27466368,"thread_start_us":558836,"threads_started":2,"wal-append.queue_time_us":721143,"wal-append.run_cpu_time_us":1096,"wal-append.run_wall_time_us":1055110}
I20260709 13:01:32.557982  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:32.564301  9448 rpcz_store.cc:276] 0709 13:01:30.918818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.918951 (+   133us) service_pool.cc:224] Handling call
0709 13:01:32.411600 (+1492649us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:32.560086 (+148486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8091520}
I20260709 13:01:32.564970  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1763) took 1361 ms. Trace:
I20260709 13:01:32.492364 13166 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.522677  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2106) took 1903 ms. Trace:
I20260709 13:01:32.567736  8889 rpcz_store.cc:276] 0709 13:01:30.619609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.826149 (+1206540us) service_pool.cc:224] Handling call
0709 13:01:32.388968 (+562819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.522311 (+133343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.522555 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.511044 12976 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.523128  8749 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:32.524430  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.524076  8743 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:32.552059  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:32.550776  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 873) took 1629 ms (client timeout 1898 ms). Trace:
W20260709 13:01:32.571381  9447 rpcz_store.cc:269] 0709 13:01:30.921609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.921740 (+   131us) service_pool.cc:224] Handling call
0709 13:01:31.850826 (+929086us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:32.550710 (+699884us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20089088}
I20260709 13:01:32.523274  8898 rpcz_store.cc:276] 0709 13:01:30.826540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.415690 (+1589150us) service_pool.cc:224] Handling call
0709 13:01:32.416143 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.418743 (+  2600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.419035 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.553011  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.573264  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.573478 13166 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.573915  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.574447  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.574627  8749 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:32.575225  8745 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:32.575564  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.575822 13166 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.576258 13158 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:32.576473 13166 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.576860 13158 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:32.561817 13159 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.577793 13159 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.576261  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:32.521090 13164 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:32.550433  8817 rpcz_store.cc:276] 0709 13:01:31.231356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.553103 (+321747us) service_pool.cc:224] Handling call
0709 13:01:32.510990 (+957887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.582640  8749 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:32.512202 13032 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.031s	sys 0.044s
I20260709 13:01:32.550302  8893 rpcz_store.cc:276] 0709 13:01:28.277327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.153412 (+2876085us) service_pool.cc:224] Handling call
0709 13:01:31.154117 (+   705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.159664 (+  5547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.159724 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.160376 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.287475 (+127099us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:31.287763 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.288136 (+   373us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.851224 (+563088us) raft_consensus.cc:1685] finished
0709 13:01:31.851250 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.851380 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30793,"raft.queue_time_us":139,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3107,"thread_start_us":79943,"threads_started":1,"wal-append.queue_time_us":452300}
I20260709 13:01:32.576261  8745 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:31.783658 13122 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
I20260709 13:01:32.565475  8821 rpcz_store.cc:276] 0709 13:01:31.203620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.557727 (+1354107us) service_pool.cc:224] Handling call
0709 13:01:32.564913 (+  7186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.592574  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:32.558552  8890 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:32.566069  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:32.504715 12975 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.524142 13083 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.570528  8744 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:32.516465 13031 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:32.584030  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:32.586047  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:32.604298  9442 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.541081 12889 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:32.523738 13082 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.590935  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:32.540272 13149 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.593595  8820 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.601750  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 694) took 1047 ms. Trace:
I20260709 13:01:32.606995  8817 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 9: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:32.521760 12751 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.513090 13157 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.581638 13160 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.645224 12975 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.647467  8890 rpcz_store.cc:276] 0709 13:01:31.554138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.516748 (+962610us) service_pool.cc:224] Handling call
0709 13:01:32.549569 (+ 32821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.601432 (+ 51863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.601686 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.589907 13032 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.031s	sys 0.045s
I20260709 13:01:32.587705 13164 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:32.647249  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:32.557637 12622 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.557500 13030 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:32.654397 13144 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:01:32.591043 13023 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:01:32.610112 12889 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:32.601819 13165 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:01:32.566062 13155 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.570255 13154 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.677361  9449 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:32.565853 13121 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.673892 12889 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.577797 13158 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.679687 13023 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:32.671754 13031 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:32.686360 13163 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:01:32.658785 13160 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.662189 13149 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.683395 13088 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:32.691531 13131 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.692759 13171 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:32.697233 13171 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:32.698196 13171 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.699091 13149 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.698848 13117 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.695932 13175 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.690104 13164 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.702422 13175 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.702796 13155 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.703198 13164 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.704487 13175 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.693517 13174 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:32.705950 13174 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.706080 13157 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.707861 13174 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.712239 13160 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.757148  8740 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:32.759197  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1758) took 4367 ms. Trace:
I20260709 13:01:32.791586 12751 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.797976 12850 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:32.799199 12850 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:32.701503 13121 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.828284 12850 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.830763 13031 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:32.833817  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 416572us
I20260709 13:01:32.835870  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:32.841774  8823 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10. Re-sending same reply.
I20260709 13:01:32.856185 13166 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.856879  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1756) took 1664 ms. Trace:
I20260709 13:01:32.860523 13171 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.858572  8749 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:32.859534  8821 rpcz_store.cc:276] 0709 13:01:31.192514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.592496 (+1399982us) service_pool.cc:224] Handling call
0709 13:01:32.592756 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.592948 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:32.646331 (+ 53383us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.855923 (+209592us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.856194 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.856258 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:32.856352 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.856462 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.856672 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27465,"prepare.queue_time_us":55494,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":32305,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":265,"thread_start_us":180578,"threads_started":1}
W20260709 13:01:32.857424  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48353.3us
I20260709 13:01:32.861495 13171 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.864470  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:32.859122 12622 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.865427 13171 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:32.865787 12987 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.866278 13171 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.828507 13088 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.867122 13030 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.867308 13121 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:32.867818 13121 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:32.868058 13030 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:32.868278 13121 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.868769 13030 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:32.873144  8817 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:32.874055  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1757) took 1681 ms. Trace:
I20260709 13:01:32.874444  8817 rpcz_store.cc:276] 0709 13:01:31.192755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.650488 (+1457733us) service_pool.cc:224] Handling call
0709 13:01:32.872630 (+222142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.873735 (+  1105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.873969 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.860699  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.878105  8817 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:32.889390  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.888032  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1761) took 1690 ms. Trace:
I20260709 13:01:32.891743  8817 rpcz_store.cc:276] 0709 13:01:31.195579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.875373 (+1679794us) service_pool.cc:224] Handling call
0709 13:01:32.875813 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.885160 (+  9347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.885460 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.867200 13171 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.759785  8741 rpcz_store.cc:276] 0709 13:01:28.392177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.989854 (+1597677us) service_pool.cc:224] Handling call
0709 13:01:30.006890 (+ 17036us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:30.007379 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.007410 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:30.014275 (+  6865us) log.cc:844] Serialized 52 byte log entry
0709 13:01:31.045556 (+1031281us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:31.143042 (+ 97486us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.351845 (+208803us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b180047f440. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:01:31.629648 (+277803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.632118 (+  2470us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.782759 (+150641us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:32.758467 (+975708us) raft_consensus.cc:1685] finished
0709 13:01:32.758526 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.758894 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163409,"prepare.queue_time_us":102730,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2433,"raft.queue_time_us":252173,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":25554,"spinlock_wait_cycles":16169088,"thread_start_us":1198679,"threads_started":2,"wal-append.queue_time_us":1515989}
W20260709 13:01:32.894992  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 340646us
I20260709 13:01:32.907523  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:32.908916  9443 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:32.910034  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:32.910701  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2108) took 2290 ms (client timeout 1878 ms). Trace:
I20260709 13:01:32.911010  9444 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:32.911208  8891 rpcz_store.cc:269] 0709 13:01:30.620213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.710295 (+ 90082us) service_pool.cc:224] Handling call
0709 13:01:32.910217 (+2199922us) spinlock_profiling.cc:243] Waited 69.2 ms on lock 0x7b5400047ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:32.910415 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159148800}
I20260709 13:01:32.913708  8896 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:32.914217  9447 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:32.915277  8815 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:32.919415 13080 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:32.919936 13080 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:32.921386 13080 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.923931 13080 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.903115 13171 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:32.938581  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 696) took 1702 ms. Trace:
I20260709 13:01:32.944154  8815 rpcz_store.cc:276] 0709 13:01:31.228755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.875800 (+1647045us) service_pool.cc:224] Handling call
0709 13:01:32.876627 (+   827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.915786 (+ 39159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.930983 (+ 15197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.915366 13174 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:32.945880 13174 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:32.946616  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2420) took 1458 ms. Trace:
I20260709 13:01:32.947135  8815 rpcz_store.cc:276] 0709 13:01:31.488490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.945235 (+1456745us) service_pool.cc:224] Handling call
0709 13:01:32.945829 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.946235 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.946262 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.946335 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:32.946365 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.946423 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.946536 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.944787 13180 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:01:32.944896  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:32.953166 13092 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:32.691664 13023 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.956357 13023 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:32.956576  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:32.959741 13166 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:32.948612 13174 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:32.949193  8815 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:32.972731  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 564259us
I20260709 13:01:32.973176  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:32.974381  8898 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:32.974279  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:32.972785  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 870) took 2055 ms. Trace:
I20260709 13:01:32.983028  9449 rpcz_store.cc:276] 0709 13:01:30.917585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.926550 (+  8965us) service_pool.cc:224] Handling call
0709 13:01:32.646999 (+1720449us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:32.972376 (+325377us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54004d1054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:32.972582 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58092544}
I20260709 13:01:32.998283  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:33.002324 13145 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.453s	user 0.006s	sys 0.014s
I20260709 13:01:33.010576 13154 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:33.013674  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.015297 13159 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:33.017680 13173 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.447s	user 0.002s	sys 0.000s
W20260709 13:01:33.018879  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.020888 13023 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:33.022087  9445 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:33.022333  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 699) took 2491 ms. Trace:
I20260709 13:01:33.022449 13023 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:33.022918 13023 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:33.023682  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.024861  9440 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:33.022713  8748 rpcz_store.cc:276] 0709 13:01:30.530872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.711403 (+180531us) service_pool.cc:224] Handling call
0709 13:01:30.711682 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.746511 (+ 34829us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:31.224432 (+477921us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.438752 (+214320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.438806 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:31.438848 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.778448 (+339600us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b1800479020. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:33.021806 (+1243358us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:33.021887 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.022122 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298844,"prepare.queue_time_us":278668,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":188200,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":29065,"spinlock_wait_cycles":94253568,"thread_start_us":433228,"threads_started":2,"wal-append.queue_time_us":327344,"wal-append.run_cpu_time_us":1380,"wal-append.run_wall_time_us":1029765}
I20260709 13:01:33.025821  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2415) took 2337 ms. Trace:
I20260709 13:01:33.026270  8818 rpcz_store.cc:276] 0709 13:01:30.688269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.910205 (+221936us) service_pool.cc:224] Handling call
0709 13:01:30.937589 (+ 27384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.958998 (+ 21409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.959067 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.959146 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:30.959214 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.025263 (+2066049us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:33.025400 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.025622 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48315136}
I20260709 13:01:33.028326  8818 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:33.029546  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2336) took 4776 ms. Trace:
I20260709 13:01:33.029975  8896 rpcz_store.cc:276] 0709 13:01:28.252973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.824724 (+2571751us) service_pool.cc:224] Handling call
0709 13:01:32.913141 (+2088417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.914222 (+  1081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.029218 (+114996us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400047ed0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:33.029408 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4803381120}
I20260709 13:01:33.032970  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2417) took 2108 ms. Trace:
I20260709 13:01:33.034523 12838 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.040257 12838 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:33.041719  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 177575us
I20260709 13:01:33.042737 13075 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.043239  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:33.037932 12807 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.039451  8822 rpcz_store.cc:276] 0709 13:01:30.924835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.016650 (+ 91815us) service_pool.cc:224] Handling call
0709 13:01:33.031802 (+2015152us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:33.032166 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.032505 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.032529 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.032593 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:33.032620 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.032675 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.032787 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10849152}
I20260709 13:01:33.041697 12838 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.049639 12838 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.051801 13027 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.053472  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.054468  8891 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.056960 13117 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:33.059916  8816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 702) took 1828 ms (client timeout 1565 ms). Trace:
W20260709 13:01:33.060472  8816 rpcz_store.cc:269] 0709 13:01:31.231068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.251098 (+ 20030us) service_pool.cc:224] Handling call
0709 13:01:33.059467 (+1808369us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400075654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:33.059664 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182148352}
I20260709 13:01:33.062152 13030 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.066659  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.068571  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.068786  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.066813  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.069787  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.070555  8744 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.070590  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.071446  8741 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:33.071352  8749 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:33.071627  8743 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:33.072953 13015 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:33.073390 13015 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:33.073550 13140 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.074568 13015 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.075871 13015 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:33.077833  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.078402 13113 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.078970 13113 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:32.990574 13166 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:33.080963 12783 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.083253 12783 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.083508 13166 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.088280 12936 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.092373 12732 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.095268  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2414) took 2407 ms. Trace:
W20260709 13:01:33.099648  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2333) took 5067 ms (client timeout 3895 ms). Trace:
W20260709 13:01:33.100147  8897 rpcz_store.cc:269] 0709 13:01:28.031830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.882601 (+850771us) service_pool.cc:224] Handling call
0709 13:01:33.099152 (+4216551us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54003bdb54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:33.099421 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4810730240}
I20260709 13:01:33.101580 13140 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.102342 13140 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.095803  8819 rpcz_store.cc:276] 0709 13:01:30.687921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.896270 (+208349us) service_pool.cc:224] Handling call
0709 13:01:30.910142 (+ 13872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.163839 (+253697us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:32.434862 (+1271023us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.858588 (+423726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.858690 (+   102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:32.858820 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.094789 (+235969us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540028bc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:33.094839 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.095057 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":688783,"prepare.queue_time_us":1225719,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":5220,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"spinlock_wait_cycles":582899584,"thread_start_us":418535,"threads_started":1,"wal-append.queue_time_us":262132}
I20260709 13:01:33.105321 13113 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.106999 13023 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.110312 13121 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.111864 13023 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.112775 13121 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.121374  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:33.126513  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 152482us
I20260709 13:01:33.128907 12997 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.132930  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:33.154615  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.159564  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:33.159723  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58822.2us
I20260709 13:01:33.171936  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 707) took 1858 ms. Trace:
I20260709 13:01:33.172128 13113 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.179039  9443 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.175679  8647 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 10, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:33.171936  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.186403  9444 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:33.186278  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 143248us
I20260709 13:01:33.165364 12997 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.174297  8824 rpcz_store.cc:276] 0709 13:01:31.280630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.922329 (+1641699us) service_pool.cc:224] Handling call
0709 13:01:32.923080 (+   751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.923676 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:32.924171 (+   495us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.135359 (+211188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.135412 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:33.135456 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.135530 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.138279 (+  2749us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":926,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":146149,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":19867,"thread_start_us":56568,"threads_started":1}
W20260709 13:01:33.189623  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 233372us
I20260709 13:01:33.191354  9449 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:33.193635  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.198729  9447 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 9 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:33.187237 13174 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.199719  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78695.6us
I20260709 13:01:33.203555  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:33.204625  8820 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.001s
I20260709 13:01:33.208382 13164 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.208907 13164 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.209333 13164 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.213184 13184 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.214092 13184 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:33.215443 12976 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.645s	user 0.006s	sys 0.009s
I20260709 13:01:33.216746 13184 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.221581  8747 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:33.221640  8746 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:33.223717  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.223966  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.224499  8824 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:33.224601  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.225037  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.225543 13109 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.226291  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.226183 13109 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.228838 13109 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.229465 13121 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.230185 13121 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.232642 13121 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.233347 13153 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.233814  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1765) took 2001 ms. Trace:
I20260709 13:01:33.233961 13153 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.234282  8817 rpcz_store.cc:276] 0709 13:01:31.232213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.903119 (+1670906us) service_pool.cc:224] Handling call
0709 13:01:32.986478 (+ 83359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.233422 (+246944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.233456 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.233526 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:33.233563 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.233628 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.233753 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":219387}
I20260709 13:01:33.235700  8742 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:33.239131  8816 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:33.240267 12759 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.240442  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.239163  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:33.240869 12759 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.241817  8824 raft_consensus.cc:2376] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12. Re-sending same reply.
I20260709 13:01:33.241436 12997 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.242854  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.243113 12759 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.243571  8824 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:33.244483  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.245579  8816 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:33.248001  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:33.245237  8824 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:33.250135  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:33.250397  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:33.251035  8816 raft_consensus.cc:2376] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12. Re-sending same reply.
I20260709 13:01:33.238226 12889 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.253423  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.253633 13038 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:33.254340 13038 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:33.255486 13038 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.259068 12889 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.260205  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:33.261189 13088 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:33.261845 13088 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.262593 13174 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.263105 13088 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.261139  8815 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14. Re-sending same reply.
I20260709 13:01:33.269783 13031 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.270349  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:33.270499 13031 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.272460  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:33.274507  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.275158 12976 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:33.275988 12976 consensus_queue.cc:1243] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:33.276561 12976 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:33.276925 12976 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:33.277362 13080 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:33.278081  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:33.278479  8820 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:33.279090  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 704) took 2027 ms. Trace:
I20260709 13:01:33.279418  8820 rpcz_store.cc:276] 0709 13:01:31.253405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.562616 (+1309211us) service_pool.cc:224] Handling call
0709 13:01:33.279001 (+716385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:33.279666 12807 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:33.280233 12807 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:33.280472  8823 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:33.280953  8817 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:33.244534 12997 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:33.277273  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.286957  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.281340 12807 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.290192  8744 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:33.289855  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.291458  8891 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:33.273061 13031 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.293995  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1760) took 2098 ms. Trace:
I20260709 13:01:33.294822  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:33.295027 12732 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:33.302812  8815 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:33.296416 13188 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.297055  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.302734  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.295629 13140 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.301167  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.279666 12759 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.306190  8820 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
W20260709 13:01:33.304826  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.304811  8816 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:33.301425  8821 rpcz_store.cc:276] 0709 13:01:31.195326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.861774 (+1666448us) service_pool.cc:224] Handling call
0709 13:01:32.866065 (+  4291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.866573 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:32.889129 (+ 22556us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.293740 (+404611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.293795 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:33.293825 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.293867 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.293952 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98302,"prepare.queue_time_us":23929,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":2227,"raft.queue_time_us":101999,"thread_start_us":302172,"threads_started":1,"wal-append.queue_time_us":370513}
I20260709 13:01:33.311947  9444 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:33.313009  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.312763 13184 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.316236 13184 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.316288 13015 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:33.317883 13184 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:33.318699  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 141977us
I20260709 13:01:33.320560 13174 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.322664 13174 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:33.325698 13103 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.326277 13103 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.327417 13103 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.330495  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:33.330603 12976 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.332258 12976 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.332077 13171 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:33.329010 13103 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:33.338810  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.308259 12759 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:33.333318 13171 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:33.350690  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1759) took 2157 ms. Trace:
I20260709 13:01:33.349179  9443 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:33.351136  8823 rpcz_store.cc:276] 0709 13:01:31.195063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.844460 (+1649397us) service_pool.cc:224] Handling call
0709 13:01:33.280213 (+435753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.280954 (+   741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.348443 (+ 67489us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54002297d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:33.350631 (+  2188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000409344}
I20260709 13:01:33.341404  9442 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:33.361897 12759 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.363451 12759 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:33.349977 13104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.280s	user 0.011s	sys 0.036s
W20260709 13:01:33.365535 13104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.461s	user 0.012s	sys 0.037s
I20260709 13:01:33.365888 13104 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.366097 13031 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:33.309407 13188 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:33.367547 13083 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.005s
I20260709 13:01:33.348434 13015 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.369771 13171 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.370270  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:33.374130  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.371764  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:33.340696 12997 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.388787  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.392920  9447 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:01:33.388595  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.390890  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.395972  9442 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:33.382993 12783 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:33.400139  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.402014 13023 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.402184 13083 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.402658 13023 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.402818 13083 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.405230 13083 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.405407 12783 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.405908 12783 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.407814  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:33.408429 13153 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.409241  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:33.410980  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81337.1us
I20260709 13:01:33.374629 12889 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:01:33.408475  8820 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.405055 13023 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:33.417495  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 138177us
I20260709 13:01:33.419250  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:33.423637  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.424894 13083 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.425464 13083 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.428246 13083 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.395658 12997 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.431707 12997 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.432185 13113 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:33.433430  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.437450  9448 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:33.438105  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.444479  9447 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:33.444752  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.445595  8891 raft_consensus.cc:2376] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10. Re-sending same reply.
I20260709 13:01:33.438541  8819 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:33.442461  9443 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:33.385689  8647 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: config changed from index 4 to 7, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129), VOTER 29c7934ee528454785432732d01ed0a0 (127.7.117.130), VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131), VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:33.421893  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.448203  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:33.448997  9449 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:33.450210  9444 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:33.420710 13038 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.451507 13038 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:33.451884 13038 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.454761  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:33.458441 13171 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.445245  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.464465 13113 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:33.465153  8789 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.661s (SENT)
I20260709 13:01:33.465623 13113 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:33.466760  8789 leader_election.cc:336] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.706s (SENT)
I20260709 13:01:33.466895 13113 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:33.467953  8789 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.833s (SENT)
I20260709 13:01:33.468405  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:33.409068 13104 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:33.439941 13188 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.474924  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.475996  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.476778 13109 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.477556 13109 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:33.477331  8647 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.478149 13109 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:33.479156  8647 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 10 to 11, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 11 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:33.479449 13109 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.481421 13121 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.481915 13121 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.386700 13164 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.478166 13185 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.483572 13185 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.484406 13164 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.482292 13121 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.484715 13185 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.486245 13185 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.490048 13140 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.490085 12237 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:33.490793 12237 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:33.490913 13140 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.491837 12237 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.493636 12237 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:33.494608 13199 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 13:01:33.499429  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.500006  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:33.500123  8891 raft_consensus.cc:2376] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13. Re-sending same reply.
I20260709 13:01:33.500837 13153 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.501233  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.501840  8890 raft_consensus.cc:2376] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:33.501258 13153 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.502420 13153 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.503724  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.504388  8897 raft_consensus.cc:2376] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10. Re-sending same reply.
I20260709 13:01:33.504468  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.505754  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.506327  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:33.510197  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.510718  8787 connection.cc:256] RPC call timeout handler was delayed by -1.4069992529999809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:01:33.452661 12783 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:33.514539  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.464339  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.480847  8647 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.518608 13164 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.518385 13153 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.519269 13164 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.520845 13153 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.515565  8820 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
W20260709 13:01:33.464202  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45419.7us
I20260709 13:01:33.523808  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.524281  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:33.557624  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:33.559309  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:33.560169  9442 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.561638  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:33.564391  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 96518.7us
W20260709 13:01:33.593770  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 151261us
I20260709 13:01:33.526412 12783 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:33.605013  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43504.4us
I20260709 13:01:33.608191  8891 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:33.605774  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.606240  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:33.612353  8643 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 270) took 30810 ms (client timeout 15000 ms). Trace:
I20260709 13:01:33.616106 13114 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.616633 13114 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.616452 13038 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.617053 13114 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.617479  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:33.625713  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.626590  8897 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:33.627612  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.628270  8897 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:33.629762  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.610634  8647 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.629225  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.634481  8897 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:33.607657 13103 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:33.629354 13075 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.651201  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.648554  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.637949 13184 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.662040  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.665701  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.641660 13103 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.702082 13103 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.657140 13180 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.706120 13180 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:33.708859 13180 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.646360  8647 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.669178 13166 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:33.668988 13184 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:33.669849  8712 connection.cc:256] RPC call timeout handler was delayed by -1.7782799880000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3895s for a response
I20260709 13:01:33.672777  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.673035  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.673722 13026 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.674381  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.674618  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.675139  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.675866  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.676576  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.738409  8817 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:33.677371  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.677767  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:33.650681 13075 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:33.678411  8822 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:33.678575  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:33.679102 12936 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:33.679361  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.679569 12783 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.679939  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.679595  9442 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:33.739219  8821 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:33.641075 13164 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:33.686213 13114 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.666621  8897 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 8.
I20260709 13:01:33.687059 13140 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:33.701181  8895 raft_consensus.cc:1217] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-12.7]   Dedup: 8.5->[10.6-12.7]
I20260709 13:01:33.747224  8895 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: [8.5-12.7]
I20260709 13:01:33.709823  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.649221  8893 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:33.716643 13174 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:33.721597  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.731624  8890 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:33.737926  8819 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:33.681630  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:33.753494  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 92278.1us
I20260709 13:01:33.736245 12732 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.752226  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 73745.8us
I20260709 13:01:33.755627  8823 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:33.755158  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.760576  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.742460 13075 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.769896  8896 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 11.
I20260709 13:01:33.770936  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.762360  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:33.770354  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:33.777405  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.761621  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.753415 13031 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.781065  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103427us
I20260709 13:01:33.751971 12759 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.777716  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.779518  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:33.851223  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.819612  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.794904  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:33.612813  8643 rpcz_store.cc:269] 0709 13:01:02.786296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.427904 (+641608us) service_pool.cc:224] Handling call
0709 13:01:03.842210 (+414306us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b5400002210. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33ab0b2 00007f90fc324a89 00007f90fc2bcda6 00007f90fc28e088 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc
0709 13:01:33.564578 (+29722368us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:23.776259 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:23.806333 (+ 30074us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:23.806401 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:01:23.806442 (+    41us) tablet.cc:662] Decoding operations
0709 13:01:23.807991 (+  1549us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:23.808090 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:23.808131 (+    41us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:01:24.007184 (+199053us) tablet.cc:701] Row locks acquired
0709 13:01:24.007235 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:01:24.007432 (+   197us) write_op.cc:270] Start()
0709 13:01:24.007602 (+   170us) write_op.cc:276] Timestamp: P: 1783602084007352 usec, L: 0
0709 13:01:24.007625 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:01:24.008386 (+   761us) log.cc:844] Serialized 5540 byte log entry
0709 13:01:25.384821 (+1376435us) op_driver.cc:464] REPLICATION: finished
0709 13:01:26.026083 (+641262us) write_op.cc:301] APPLY: starting
0709 13:01:26.026230 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:26.027738 (+  1508us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:26.027889 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:26.193862 (+165973us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:26.194257 (+   395us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=26
0709 13:01:26.194301 (+    44us) write_op.cc:312] APPLY: finished
0709 13:01:26.194757 (+   456us) log.cc:844] Serialized 95 byte log entry
0709 13:01:26.464388 (+269631us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.502870 (+ 38482us) write_op.cc:454] Released schema lock
0709 13:01:26.502956 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:26.503338 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.503372 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":65988,"reported_tablets":48,"spinlock_wait_cycles":80463104,"child_traces":[["op",{"apply.queue_time_us":640855,"apply.run_cpu_time_us":24234,"apply.run_wall_time_us":477295,"num_ops":13,"prepare.queue_time_us":315155,"prepare.run_cpu_time_us":65998,"prepare.run_wall_time_us":450660,"raft.queue_time_us":562811,"raft.run_cpu_time_us":14420,"raft.run_wall_time_us":466909,"replication_time_us":1354476,"thread_start_us":1165783,"threads_started":5,"wal-append.queue_time_us":732484,"wal-append.run_cpu_time_us":18214,"wal-append.run_wall_time_us":2973512}]]}
I20260709 13:01:33.834995  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.781347  8647 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 9, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.864032  8896 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:33.869925  8647 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 5 to 9, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.807801  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.814884  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.871845  8747 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:33.817667  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.858855  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.875149  8746 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:33.814102  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.824559  8864 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:33.876705  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52228.9us
W20260709 13:01:33.806061  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 294329us
I20260709 13:01:33.806581  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:33.877532  8749 raft_consensus.cc:2368] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13.
I20260709 13:01:33.817242  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.784562  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:33.879998  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.775005 13015 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.878609  8745 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14. Re-sending same reply.
I20260709 13:01:33.883112  8748 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:33.757651 12889 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.906770 12889 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.907291 12889 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.909372 13188 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.909827 13188 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.910189 13188 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.825491  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:33.820960 13023 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.884991 13194 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:33.885267 13184 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:33.758527 13026 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.774854 12936 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.785964 12783 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.896214  8647 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:33.897388 13201 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:33.899048  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:33.843209 13038 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:33.902319  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 143670us
I20260709 13:01:33.817855 12838 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.883062  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:33.762993 13166 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.915256  8889 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:33.918505  8743 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:33.919497  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:33.921912  8744 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.857844 13114 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:33.926432  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.930398  8742 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.815121 12732 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.926162  8647 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 6, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 6 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.905799 13015 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.805405 13174 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:33.933429  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:33.934486  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52463.6us
I20260709 13:01:33.934540  8741 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:33.916934 13194 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:33.913210 13031 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:33.951452  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52536.2us
I20260709 13:01:33.953881  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:33.936493 13015 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.965229  8891 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.950179  8647 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:33.970683  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:33.942317 13088 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.974229  8818 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:33.923393 13121 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.943642 13038 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:33.975397  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:33.945164 13194 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:33.978035  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 293336us
I20260709 13:01:33.974434  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:33.972874 13204 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:01:33.987107  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.999477  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 220118us
I20260709 13:01:33.976642 13088 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.942647 12838 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:33.999028  8647 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:34.003417  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.016036  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.020547  8891 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:34.025192  8744 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:34.007652 13088 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.013062 12936 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:34.026176  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 251856us
W20260709 13:01:34.044885  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 247941us
W20260709 13:01:34.045128  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.022415  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:34.006386 13171 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.047938  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.006615 13201 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:01:34.059464  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:34.060810  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.061216  8747 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:34.061617  8745 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.062428  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.063982  8747 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:34.064296  8747 pending_rounds.cc:85] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 8
I20260709 13:01:34.064527  8747 pending_rounds.cc:107] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:34.064788  8747 raft_consensus.cc:2889] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.065572  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.065678  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.066363  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:34.067667  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.067821 13026 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:34.068420  8740 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:34.068969  8647 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:34.066432  8749 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.017598 13113 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:34.074577  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60700.1us
I20260709 13:01:34.027390 13164 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:34.003998 13030 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:01:34.044313 12783 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:34.083712  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.021656 13023 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:34.026883 13080 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:34.102931  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.048884 13216 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:34.108904  8647 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:34.110754  8647 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 3 to 7, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.113739  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.113827 13153 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:34.114050 12807 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:34.115152  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.115818  8747 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:34.116773  8647 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 6 to 7, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 7 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:34.117625  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 92275.4us
I20260709 13:01:34.118818  8647 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:34.120486  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.120309  8647 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.115630 12807 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:34.121506 12838 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:34.121624 12807 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:34.123106 13207 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:01:34.122769 12807 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.123955 12807 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:34.124470  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78858us
I20260709 13:01:34.129500 13104 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:34.093391 13206 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:01:34.131639  8741 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.082360 13026 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:34.137079  9414 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.565s (SENT)
I20260709 13:01:34.138198 13174 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:34.137931 13075 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.138732 13104 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:34.164227  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.139452 12889 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.140097  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.147979 13164 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:34.149654 13166 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:34.150287 13211 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:01:34.158142  8745 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:34.150974 13103 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.093255 13113 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:34.074820 12759 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:01:34.163777  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 136515us
I20260709 13:01:34.169925  8749 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
W20260709 13:01:34.189394  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.193867 13113 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.194983  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:34.198041 13075 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:34.201118 13083 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:34.201658 13083 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:34.202713 13083 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.218017 13104 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.225319  8743 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:34.157696 13216 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:34.179569 13164 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:34.173030  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60330.3us
I20260709 13:01:34.173802 13227 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:34.161571 13031 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:34.169903 13026 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:34.169768 13188 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.151818 13171 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 13:01:34.172194  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 96784us
I20260709 13:01:34.149718 13023 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:34.519927  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 223259us
I20260709 13:01:34.234022  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:34.521561  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 54739.1us
I20260709 13:01:34.521833  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:34.287372  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.241132  8742 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:34.235813  8894 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:34.524907  9442 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:34.405519  9444 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:34.309885  8748 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:34.474329  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.327961  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:34.401206  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:34.531702  9440 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:34.533521  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 415268us
I20260709 13:01:34.543041  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:34.543884  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.293870  9414 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.689s (SENT)
I20260709 13:01:34.552556  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:34.553128  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.352087  9443 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.234146 12732 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:34.528083 13083 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.517630 13221 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.577495  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.580935  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:34.425576 13224 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.327462  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:34.283566 13121 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:34.460035 13038 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.551225 12807 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.405349 13109 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.173s
I20260709 13:01:34.425865 13201 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.522125 13227 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:34.573532  9447 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:01:34.584175  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 290398us
I20260709 13:01:34.354344 12759 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.242s
I20260709 13:01:34.518781 13185 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:34.592811  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.592507  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.471725 13194 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.576931 13031 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:34.597533  8820 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:01:34.594837  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:34.594556  8789 leader_election.cc:400] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:34.585095  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.609823  8821 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:34.336155 13015 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.613312  8789 leader_election.cc:403] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:34.616014  8817 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:34.616835 13026 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.576344 12783 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:01:34.617821 13194 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:34.619879 13219 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:01:34.620635  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.611873  8820 pending_rounds.cc:85] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 6
I20260709 13:01:34.621760  8749 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:34.605399 12889 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:34.472178 13216 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.594921 13038 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.614235 13031 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.589839 13224 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.589493 13221 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.656075  8749 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:34.630585  8820 pending_rounds.cc:107] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 9.7
I20260709 13:01:34.635907 13103 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:34.607990 13201 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.614049 13185 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.634894 12997 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.663069  8820 raft_consensus.cc:2917] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Aborting config change with OpId 9.7: Aborted: Op aborted by new leader
I20260709 13:01:34.648415 13031 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.673048  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:34.673072  8823 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:34.684877  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.685339 12759 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:34.685672  8822 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:34.686884  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.689567  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.690162  8821 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:34.687992  8820 raft_consensus.cc:2889] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.692286  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:34.692845  8824 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:34.696010  8821 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:34.703787  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:34.709436  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119520us
I20260709 13:01:34.685952 12759 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.715934  8816 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:34.716730  8816 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:34.662784 13114 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.011s	sys 0.018s
I20260709 13:01:34.702358 13015 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:34.615164 13227 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.712855  8819 raft_consensus.cc:2376] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9. Re-sending same reply.
I20260709 13:01:34.730302  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.727711  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.740370  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:34.721688  8816 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.700805 13164 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:34.708997 13228 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.702384 13038 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:34.746708  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 116626us
I20260709 13:01:34.748354  8821 raft_consensus.cc:2368] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:34.749456  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.754518  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.752981  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:34.757678  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:34.759577  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:34.761179  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:34.754032  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.765908  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.769027  8895 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:34.724625 13114 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.011s	sys 0.018s
I20260709 13:01:34.723918 13188 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:34.768302  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.772430  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:34.774371  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 202981us
I20260709 13:01:34.777464  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.778267  8889 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:34.780220 13234 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:34.780789  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:34.792167  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.794067  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.756119 13171 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.797926  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.798923  8889 raft_consensus.cc:2410] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:34.799247  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:34.801709  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77926.6us
I20260709 13:01:34.800071  8896 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:34.775022 13113 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.833921  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:34.841948  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.796451 13038 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:34.796223 13015 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.879289  8889 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:34.879760  8895 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:34.762619 13228 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:34.880333  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:34.885876  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 105210us
I20260709 13:01:34.885742  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:34.886526  8898 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:01:34.888262  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 135381us
W20260709 13:01:34.890368  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56566.3us
I20260709 13:01:34.833925 13233 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:34.889766 13219 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:34.932075  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:34.932981  8819 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:34.933357  8743 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:34.936064  8889 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:34.937824  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.939513  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:34.942137  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:34.940564  8712 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 3.895s (SENT) [suppressed 35 similar messages]
I20260709 13:01:34.889470 13038 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:34.932103 13219 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.930168 13239 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.949512  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.950500  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:34.950621  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:34.930385 13233 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:34.952013  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:35.015873  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.019078  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 701) took 1265 ms. Trace:
I20260709 13:01:35.029081  8824 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:34.968407  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:35.001142  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.006666  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 126435us
I20260709 13:01:34.968538 13244 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.970634  8889 raft_consensus.cc:2410] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:34.947063 13185 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:35.031771  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 81245.8us
I20260709 13:01:35.032138  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:35.033470  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:35.035333  8819 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:34.967836 13219 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.028271  8894 rpcz_store.cc:276] 0709 13:01:33.734944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.757016 (+ 22072us) service_pool.cc:224] Handling call
0709 13:01:34.980516 (+1223500us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540046df54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:34.999850 (+ 19334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052387072}
I20260709 13:01:34.967468 13188 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:35.013753 13088 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.932s	user 0.009s	sys 0.005s
I20260709 13:01:34.968528 13234 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.010069 13104 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.048852  8895 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:35.048772  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:34.972860 13023 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:35.028748 12889 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.973201 13247 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.032058 13201 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:34.887115 13171 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.111s
W20260709 13:01:35.052819  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.080271  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:35.019804 13221 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:35.056500  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.045609 13231 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.057067  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.007922 13239 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.036322 13194 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:35.059155 13088 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:35.059597  8749 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:35.059664 12783 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:35.045806 13217 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 13:01:35.060734  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 151018us
I20260709 13:01:35.106825  8894 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:35.106792  8787 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:35.046106  8743 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.059685  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.028828 13233 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.065529  8741 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:35.069156  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128638us
I20260709 13:01:35.094574 13171 consensus_queue.cc:1243] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:35.039393 13224 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.083921  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.007279 13109 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:35.085888  8819 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.053580  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:35.049278 12759 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.089645  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:35.084987 12889 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.092087  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.126230  8744 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:35.036898 12732 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:01:35.092633  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.129315  8897 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:35.093726  8817 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.051507 12807 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.055647 13153 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.068578 13164 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:35.099078  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61567.7us
I20260709 13:01:35.059899  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:35.156085  8823 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:35.156876 13243 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:35.108358 13255 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:01:34.981813 13038 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.110497 13240 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:35.056172 13245 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
I20260709 13:01:35.115643  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:35.126595 13015 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.067404 13185 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:35.056631  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:35.243261  8889 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:35.077513 13104 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.127434  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.127820  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.105772 13221 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.130364  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.153764  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:35.154613 13222 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.177812  8817 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:35.159127 12807 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:35.218087 13254 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:35.217167  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 175441us
I20260709 13:01:35.117413  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.088528 13247 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.064879 13244 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.128196 13030 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:35.219722  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:35.341637  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 308295us
I20260709 13:01:35.342244  8864 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:35.342832  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:35.343986  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 188117us
I20260709 13:01:35.345458  8819 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:35.345994  8824 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:35.346266  8748 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:35.349375  8743 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:35.350507  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:35.351621  8740 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:35.352695  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:35.354209  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:35.210143 13240 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.358472  9448 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 13. (index mismatch)
I20260709 13:01:35.359129  8889 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:35.359937  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:35.360695  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 280598us
I20260709 13:01:35.361402  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:35.358423  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:35.246158 13031 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.003s
I20260709 13:01:35.364116  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.353525  8894 raft_consensus.cc:2376] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
W20260709 13:01:35.246240 13083 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.666s	user 0.005s	sys 0.006s
I20260709 13:01:35.363575  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.370146  8746 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:35.370236  8745 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:35.363729  9440 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:35.366372  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:35.371296  9442 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:35.363291  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:35.369391  8742 raft_consensus.cc:2410] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:35.206933 13153 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:35.246655 13015 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.247805 13228 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:35.398558  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 261413us
I20260709 13:01:35.344882 13030 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:35.245872 13088 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:35.377372  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:35.378188  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:35.396011  8747 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:01:35.435524  8747 pending_rounds.cc:85] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:01:35.435948  8747 pending_rounds.cc:107] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:01:35.436218  8747 raft_consensus.cc:2889] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.347779 13222 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.450743 13222 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.450860 13104 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:35.451957  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73873.1us
I20260709 13:01:35.454289  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:35.344367 13221 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:35.349622 13239 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.436929  9448 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:35.437721  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65316.4us
I20260709 13:01:35.362758 13254 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:35.364895 13240 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.373621 13171 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:35.461911  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 355295us
I20260709 13:01:35.495399  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:35.506701  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:35.497257  8712 leader_election.cc:336] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.540s (SENT)
I20260709 13:01:35.513959  9440 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:35.514441  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:35.511106  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:35.454939 13248 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.516953  9444 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:35.577925  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.455686 12759 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:35.376670 13031 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.494156 13015 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:35.456020 13228 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:35.456307 13260 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.510152 13217 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:01:35.437629 13255 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.560779  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:35.376505 13164 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.520251 13254 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:35.565122  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67900.8us
I20260709 13:01:35.566078  8892 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:35.458703 13257 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 13:01:35.531378 13030 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:35.458276 13261 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:35.573704  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.500113 13221 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:35.580950  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71423.2us
I20260709 13:01:35.585879  8817 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:35.579385 12759 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.597033  9442 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:35.568893 13109 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:35.620872  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.576992 13234 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:35.577608 13248 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:35.621723  8817 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:35.620148  9448 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
W20260709 13:01:35.629580  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.630306  9444 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 9 LEARNER]: Advancing to term 11
I20260709 13:01:35.597340 13121 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:01:35.640484  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:35.670324 13261 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:35.945250  8820 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:35.946492  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.670089  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 711) took 1113 ms. Trace:
I20260709 13:01:35.642978  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:35.641858  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:35.944585  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 447464us
I20260709 13:01:35.946012 13261 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.618654 13255 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:35.678298  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.087423  8815 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:36.088871 13226 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.090262 13226 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.090654 13226 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:36.092640 13241 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.198s	user 0.001s	sys 0.003s
I20260709 13:01:35.933115 13222 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:36.089650  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.089579  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 464630us
W20260709 13:01:36.031308  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 388444us
I20260709 13:01:36.097661  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.098052  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:35.767856  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 146701us
W20260709 13:01:36.103832  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.650941  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.947388  8892 rpcz_store.cc:276] 0709 13:01:34.556303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.556803 (+   500us) service_pool.cc:224] Handling call
0709 13:01:35.669879 (+1113076us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b540015f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:35.670017 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829733760}
I20260709 13:01:36.056275  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:36.053146  9444 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 11 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:36.055181  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.111619  8820 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:35.960999  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:35.627075 12759 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.622458 13015 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.669548 13262 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:01:35.670864 13251 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:01:35.705793 13233 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.592s	user 0.005s	sys 0.006s
I20260709 13:01:35.679536 13109 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:35.710947 13256 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 13:01:35.669646 13257 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:36.130529  8815 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:35.961532 13030 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.781580 13265 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.131878  9447 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:35.773469 13270 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.135483  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.157096  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.157426 13284 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:36.157858  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.157961 13284 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.135744  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2360) took 1393 ms. Trace:
I20260709 13:01:35.871150 13031 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.873498 13274 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:35.710135 13153 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.033715 13217 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.642736 12783 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.145056 13121 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:36.146070  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.146807  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2356) took 1414 ms. Trace:
I20260709 13:01:36.145977 13229 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:36.148021 13228 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:36.149107 13174 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.149922 13225 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.212313 13225 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.214430 13225 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.214468  8816 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:36.136848 13233 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.196506  8890 rpcz_store.cc:276] 0709 13:01:34.732739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.793515 (+ 60776us) service_pool.cc:224] Handling call
0709 13:01:34.793758 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.803944 (+ 10186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:34.893782 (+ 89838us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.435802 (+542020us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.146415 (+710613us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.146471 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:36.146543 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.146617 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.146746 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":716467,"prepare.queue_time_us":73565,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":717034,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":7808,"thread_start_us":473582,"threads_started":2,"wal-append.queue_time_us":407802}
I20260709 13:01:36.218941 13233 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.153460 12732 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:36.153836  8820 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:35.926338 13244 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.961759 13247 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:35.941505 13275 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.349s
I20260709 13:01:36.156291 13227 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.158540 13284 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.948076 13260 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:35.667696 12889 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.117439 13038 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.190591  8818 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:36.191071  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.147298 13171 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:36.158870  8893 rpcz_store.cc:276] 0709 13:01:34.734007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.806155 (+ 72148us) service_pool.cc:224] Handling call
0709 13:01:34.878858 (+ 72703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.881393 (+  2535us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:34.947363 (+ 65970us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.373369 (+426006us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.536587 (+163218us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b180038f260. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1
0709 13:01:35.569996 (+ 33409us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b180038f260. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:36.126982 (+556986us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b180038f260. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:01:36.127169 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.127238 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:36.127307 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.127403 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.127594 (+   191us) inbound_call.cc:177] Queueing success response
0709 13:01:36.154205 (+ 26611us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b180038f260. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
Metrics: {"mutex_wait_us":25545,"prepare.queue_time_us":55760,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":15150,"raft.queue_time_us":763228,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":44988,"spinlock_wait_cycles":129867008,"thread_start_us":322739,"threads_started":1,"wal-append.queue_time_us":555269}
I20260709 13:01:36.194038  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.252600  8745 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:36.198585 13109 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:36.256265  8712 leader_election.cc:336] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.887s (SENT)
I20260709 13:01:36.207665 13023 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:01:36.257313  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 145014us
I20260709 13:01:35.884627 13234 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:36.192684 13219 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.111s	user 0.008s	sys 0.005s
I20260709 13:01:36.197068  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:36.137537 13257 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:36.197712 13262 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:36.124677 13015 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.223084 13174 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.195995 13121 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:36.224804 13228 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.276708  8715 leader_election.cc:400] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:36.231078  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:36.277276  8715 leader_election.cc:403] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:36.277691  8741 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:36.210150 13104 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.237762  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.253731  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:36.259203  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 101437us
I20260709 13:01:36.198038 13217 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.219659 13233 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:01:36.262805  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65845.6us
I20260709 13:01:36.191721 13265 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.209980 13031 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.811322 13248 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.198037 13270 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.272899 12889 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:36.288059 13104 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.311848 13244 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.332993 13244 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.210911 13229 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.335423 13229 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:36.229461  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90049.1us
I20260709 13:01:36.335912 13244 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.209206 13274 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.338754 13274 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.290767  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:36.339727  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49076.4us
I20260709 13:01:36.340461  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:36.340529 13255 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.341668  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.342474 13113 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:36.343156 13113 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.343753 13113 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:36.346040 13171 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.348887 12807 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.327400 13031 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.279692 13121 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:36.305624  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.312369 13288 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.318140  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1795) took 1206 ms. Trace:
I20260709 13:01:36.357796 13288 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.358500  8747 rpcz_store.cc:276] 0709 13:01:35.111740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.113038 (+  1298us) service_pool.cc:224] Handling call
0709 13:01:35.113728 (+   690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.450426 (+336698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.317832 (+867406us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400358c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:36.318039 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603396352}
I20260709 13:01:36.290347 13265 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.307591 13257 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:36.295800 13276 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:36.360357 13288 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.361099 13257 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:36.362358 13228 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.363359 13222 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:36.306221 13283 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:01:36.300715 13260 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:36.296968 13280 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:01:36.297706 13282 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:36.369640  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:36.369848 13282 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:36.370359 13282 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.371577 13031 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:36.371666 13286 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:36.372259 13286 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:36.309602 13266 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.513s	user 0.005s	sys 0.000s
I20260709 13:01:36.290427 13217 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.373845  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 715) took 1338 ms. Trace:
I20260709 13:01:36.271970 13219 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.374687 13272 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:36.375169 13272 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:36.318639 12759 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.376132 13219 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:36.376269 13272 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.377003  9449 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:36.329993 12889 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:36.377848 12889 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.374239  8891 rpcz_store.cc:276] 0709 13:01:35.035189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.049947 (+ 14758us) service_pool.cc:224] Handling call
0709 13:01:35.059219 (+  9272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.335990 (+276771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.336028 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.349939 (+ 13911us) log.cc:844] Serialized 26 byte log entry
0709 13:01:35.544114 (+194175us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:35.544187 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.544293 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.308608 (+764315us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:36.373588 (+ 64980us) raft_consensus.cc:1685] finished
0709 13:01:36.373609 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.373791 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4799,"raft.queue_time_us":6417,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10126,"spinlock_wait_cycles":277614464,"thread_start_us":441635,"threads_started":2,"wal-append.queue_time_us":258294}
W20260709 13:01:36.381034  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96341.2us
W20260709 13:01:36.360500 13240 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.844s	user 0.003s	sys 0.009s
I20260709 13:01:36.271539  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:36.399516  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 123050us
I20260709 13:01:36.311641 13015 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.400945 12807 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
W20260709 13:01:36.402824  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.418910 13270 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.419804 13272 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.405922  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 791) took 1065 ms. Trace:
I20260709 13:01:36.421787  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:36.422194  9447 rpcz_store.cc:276] 0709 13:01:35.340469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.340619 (+   150us) service_pool.cc:224] Handling call
0709 13:01:36.405724 (+1065105us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400197654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:36.405861 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751869312}
I20260709 13:01:36.407688 13226 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:36.309110 13221 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.001s
I20260709 13:01:36.408355  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.410207  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.431154  8817 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:36.373344 13286 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.414821  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 793) took 1073 ms. Trace:
W20260709 13:01:36.434921  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48217.3us
I20260709 13:01:36.435407  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.427510  8823 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:36.435110  9449 rpcz_store.cc:276] 0709 13:01:35.341163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.341272 (+   109us) service_pool.cc:224] Handling call
0709 13:01:35.362961 (+ 21689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.377456 (+1014495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.414574 (+ 37118us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54006dfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:36.414732 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332245888}
I20260709 13:01:36.415766 13216 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:36.437867  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.379462 13166 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:36.377146 12759 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.441246  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:36.443477  8890 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:36.446251  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:36.446947  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:36.447593  9448 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.447152  9444 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:36.449010  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:36.450557  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:36.517243  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:36.517274  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61727.3us
I20260709 13:01:36.456979  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:36.456090  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:36.519501  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 119861us
I20260709 13:01:36.520571  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.433624 13248 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.295886 13278 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.243s
I20260709 13:01:36.523307  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.525069  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.526937  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:36.528368  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:36.534955  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78078.4us
I20260709 13:01:36.530146  8890 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:36.434849 12807 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.438434 13217 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.440148 13254 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.438716 13031 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.438789 13219 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.400570 13023 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:01:36.535099  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:36.550881  8862 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:36.552366  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.447391 13166 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:36.454545 13270 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.457819 13274 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:36.546376  9440 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:36.555882  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:36.629329  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 178890us
I20260709 13:01:36.439057 13216 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:36.522341 12732 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.568435  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.640130  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.640518  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.556522  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 320996us
W20260709 13:01:36.648013  8712 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.914s (SENT)
I20260709 13:01:36.648425  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:36.655345  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.656889  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:36.517905 13222 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:01:36.522509 13268 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:01:36.619575  8891 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
W20260709 13:01:36.620430  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 69723us
W20260709 13:01:36.623369  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 81191.4us
I20260709 13:01:36.423820 13240 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:36.624856  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:36.625885  9442 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:36.674537  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.621161 13233 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:36.626721  8823 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.518824 13265 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:36.630267  9447 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:36.631026 13201 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:36.631271 13281 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:36.633610 13253 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.634578 13202 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:01:36.555358 13286 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:36.522432 13228 raft_consensus.cc:2764] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:36.639236  9448 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:01:36.555302  9449 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:36.434398 13221 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.682638 13201 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.527871 13185 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:36.621459 13031 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.677809  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:36.746366  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 68639.3us
I20260709 13:01:36.678287  8822 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:36.533902 13225 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:01:36.681588  8819 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:36.681888  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.686193  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.748978  8897 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:36.688500  8890 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:36.693492  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.619508 13217 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.694200  8892 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:36.708781  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.710358  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:36.705403  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:36.747843  8894 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:36.706013  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:36.698287  8818 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:36.710956  8823 pending_rounds.cc:85] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:01:36.766359  8823 pending_rounds.cc:107] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:01:36.766767  8823 raft_consensus.cc:2889] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.700817  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:36.640734 13270 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.683418 13038 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.680022 13233 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:36.674716  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44588.7us
I20260709 13:01:36.659847 12732 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.770058 13038 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.771898  8893 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:36.753984  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2435) took 2082 ms. Trace:
I20260709 13:01:36.770098 13233 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:36.685699 12237 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:01:36.775903  8815 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.720666 13265 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:36.764986  9446 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:36.734238 13015 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:36.743659 13201 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.690478 13194 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:01:36.658762 13219 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.802284  8895 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:01:36.787633 13030 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.001s
W20260709 13:01:36.794921  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 146967us
I20260709 13:01:36.798229 13221 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.763293 13227 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:36.709628 13166 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:36.764583 13185 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.799039 13264 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.696914 13253 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:36.828976  9441 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:36.774755 13080 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:01:36.801348  9445 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:36.810094  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.836426  8824 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:36.813447  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.838618  8787 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:36.840313  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.856799  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.857841  8824 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:36.858958  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.786263 13278 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:01:36.871515  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:36.820071 13226 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:36.891188 13299 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 13:01:36.903820  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 63605.6us
I20260709 13:01:36.905166  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:36.865694 13221 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:36.868971 13030 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.876274 13166 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:36.924082  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 174591us
I20260709 13:01:36.877492 13228 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.865075 13227 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:01:36.900803 13023 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:36.690668 13274 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:01:36.968639  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 267966us
W20260709 13:01:36.971887  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66827.1us
I20260709 13:01:36.978047  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:36.927656 12889 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.001s
I20260709 13:01:36.926934 13109 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:36.926270 13221 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:36.999820  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.000428  9441 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:36.978601 12732 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:36.987848 13194 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:36.994472 13164 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:37.009516  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.007558  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.007890  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.010488  8748 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:37.015692  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.026440  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:37.027020  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.025781  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.027396  8746 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:37.025906  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.011735  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.027917  8741 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:37.028280  8749 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.026158  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.016880  8894 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:37.029366  8745 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:36.919440 13265 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.032028  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:36.990226 13023 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.031860  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.035390  8894 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:37.035269  8746 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:37.035915  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.008085 13194 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.038074  8743 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:37.048794  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:37.038216  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:37.038630  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.039393  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.034570 13031 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:37.044451 13257 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.003s
I20260709 13:01:37.045462 13217 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.046195 13221 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.074054 13228 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:37.074584 13228 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.076516 13228 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.993817 12889 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.076696  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:37.064145 13194 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.078001  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.078636 12889 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.011395 13109 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:37.079851 13109 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.047289 13023 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:37.047983  8748 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:37.048247  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:37.049646  8744 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:37.063158 13284 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.810s	user 0.000s	sys 0.010s
I20260709 13:01:37.051406 13270 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:37.055120 13303 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:01:37.087872 13284 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.096442 13284 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.055540  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.055845  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:37.059903 13272 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.000s
I20260709 13:01:37.060434  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:37.107837  8742 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:37.061107 13185 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:37.061451 13216 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:37.062315 13121 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
I20260709 13:01:37.063123  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.122473  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.122872  8743 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:37.108145  8890 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:37.066268  8889 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:37.069530  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.000324 13171 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:37.064375  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:37.124805  9443 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
W20260709 13:01:37.162204  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 97910.9us
I20260709 13:01:37.125671  8745 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:37.030009 13164 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:37.038004 13282 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.002s
I20260709 13:01:37.073089  8897 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:01:37.038308 13304 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:01:37.093452  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.084199 13228 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:37.171259  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.108683  8744 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:37.171703  8741 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:37.071954  8746 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:37.193750  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:37.193987  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:37.194499  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:37.195832  8892 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.197654  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:37.198454  8745 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:37.198505  8889 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:37.201895  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.202663  8823 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:37.203840  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:37.204799  8744 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.124593 13121 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.162726 13270 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:37.214741  9412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.012s	sys 0.008s
W20260709 13:01:37.219681  9412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.012s	sys 0.008s
W20260709 13:01:37.220062  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 670364us
I20260709 13:01:37.214852  8893 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.171248 13253 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:37.172279 13272 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:37.194319 13185 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:37.193459 13216 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.220885  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:37.236315  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:37.226094 13234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.011s	sys 0.013s
I20260709 13:01:37.240947 12889 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:37.245532 12889 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:37.246341 13309 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:01:37.246627 12889 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.239126 13217 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.250653 13174 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:37.253638 13164 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.176061 13282 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.232035 13185 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:37.259161 13282 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.228120 13272 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.228415 13270 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.208580 13228 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.283308  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:37.284287  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:37.268507  8892 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:37.242817 13234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.014s
I20260709 13:01:37.298789  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.300904  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.301530  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:37.301637 13104 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.290110 13268 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:37.302690  9443 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:37.301931 13228 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.304311 13270 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.263980 13185 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.307293  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.308045  9443 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:37.308079 13185 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.294668 13023 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:37.309561 13023 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.309950 13023 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.287153 13272 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.286484 13121 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.318136  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:37.294530 13257 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:37.319520  8744 raft_consensus.cc:1217] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Deduplicated request from leader. Original: 6.7->[7.8-13.11]   Dedup: 9.9->[11.10-13.11]
I20260709 13:01:37.320652  8744 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: [7.8-13.11]
I20260709 13:01:37.297410 13219 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.324759  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.325620  9443 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:37.320447 13257 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.332973 13257 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:37.338299 13248 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.793s	user 0.007s	sys 0.002s
W20260709 13:01:37.341682 13201 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:01:37.342401 13201 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.344075  8891 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:37.345367  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.345698  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.346216  9447 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:37.345376  9448 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.348239  8744 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:37.350414  9443 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:37.351454  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:37.352041  8892 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:37.355659  9442 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:37.355680  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:37.358017  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.364017  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:37.367344  9440 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:37.368541  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:37.369498  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:37.370098  8893 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:37.380815 13268 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:37.383325 13268 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.384438 13268 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:37.398138 13233 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
I20260709 13:01:37.398663 13233 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.437083 13201 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.440330  8819 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:37.439270 13166 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:01:37.425997 13254 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:37.447809 13254 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:37.472795 13233 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:37.547494 13331 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:01:37.551287 13257 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.553617 13320 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.554298 13320 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.556303 13194 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.559023  9446 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:37.556965 13320 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:37.571883 13281 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.876s	user 0.010s	sys 0.002s
I20260709 13:01:37.573575 13254 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:37.575976 13333 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:01:37.577914 13254 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.578588 13281 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:37.580346 13257 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.594269 13272 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.594452 13023 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.598486  9449 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:37.602542  8748 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:37.603133 13194 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.604224 13023 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:37.607259  8740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.040s	sys 0.050s
W20260709 13:01:37.607631  8740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.040s	sys 0.050s
I20260709 13:01:37.612702 13272 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.615250 13268 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.620232 13281 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.620864 13281 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.621963 13268 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.622982 13281 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.631783 13281 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.632309 13281 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.637042 13281 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:37.660820  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.672022 13185 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:01:37.674531  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1802) took 2609 ms (client timeout 2450 ms). Trace:
W20260709 13:01:37.674926  8821 rpcz_store.cc:269] 0709 13:01:35.065342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.083046 (+ 17704us) service_pool.cc:224] Handling call
0709 13:01:37.674373 (+2591327us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b5400076054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:37.674461 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5861740800}
I20260709 13:01:37.675904  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.676725  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.677161  8821 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:37.677888  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.678678  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.679088  8821 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:37.679728  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.680563  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.681809  8821 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:37.705039 13038 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:37.714404  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.714684  8821 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:37.715037  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:37.723119  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2378) took 1038 ms. Trace:
I20260709 13:01:37.723610  8891 rpcz_store.cc:276] 0709 13:01:36.684228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.684427 (+   199us) service_pool.cc:224] Handling call
0709 13:01:37.722857 (+1038430us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400094cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:37.723041 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333504256}
W20260709 13:01:37.725232  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.749841 13202 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:37.750355 13202 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:37.750974 13185 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.751453 13185 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.751273 13202 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.752450 13202 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.753675 13185 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.758267 13026 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:37.758741 13026 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:37.759605 13026 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.761421 13026 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:37.804131 13306 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:37.804746 13306 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:37.809821 13306 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.811043 13306 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:37.841267  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2458) took 1032 ms. Trace:
I20260709 13:01:37.841789  8824 rpcz_store.cc:276] 0709 13:01:36.809184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.871223 (+ 62039us) service_pool.cc:224] Handling call
0709 13:01:37.841012 (+969789us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540033bdd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:37.841204 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697836288}
W20260709 13:01:37.868253  8712 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.004s	sys 0.016s
W20260709 13:01:37.868817  8712 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.005s	sys 0.017s
W20260709 13:01:37.869129  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.05107e+06us
W20260709 13:01:37.869740  8712 leader_election.cc:336] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.135s (SENT)
I20260709 13:01:37.870088  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:37.876015  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.876376  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:37.880249  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.880642  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.882194  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.882493  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:37.884588  8823 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:01:37.876622  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.880364  8712 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.403s (SENT)
I20260709 13:01:37.885555  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:37.886864  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:37.890702  8712 connection.cc:256] RPC call timeout handler was delayed by -1.6284480150000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 13:01:37.890868  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 899) took 1444 ms. Trace:
I20260709 13:01:37.891330  9441 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.13-11.13]
I20260709 13:01:37.891330  9445 rpcz_store.cc:276] 0709 13:01:36.446524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.447506 (+   982us) service_pool.cc:224] Handling call
0709 13:01:37.890674 (+1443168us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540065f4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:37.890804 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392168320}
W20260709 13:01:37.891445  8712 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.423s (SENT)
W20260709 13:01:37.892385  8749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.020s	sys 0.015s
W20260709 13:01:37.893584  8749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.020s	sys 0.015s
I20260709 13:01:37.893102  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:37.906620  8863 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.638s (SENT)
I20260709 13:01:37.907196  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:37.909957 13346 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:37.910229  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.912998 13346 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:37.913255 13347 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:37.914469  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3413480149999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
I20260709 13:01:37.914336 13347 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:37.915133  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.915612  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.913717 13346 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:37.916025 13346 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.916105  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1901480150000907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260709 13:01:37.916622 13346 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:37.916822 12759 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.917412 12759 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.919461 12759 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:37.920636 12759 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:37.921427 13347 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.922545 13347 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.922983 13347 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.921084 12759 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:37.925448 13347 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.925626 13346 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:37.926900 12759 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:36.772645  8816 rpcz_store.cc:276] 0709 13:01:34.671672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.671783 (+   111us) service_pool.cc:224] Handling call
0709 13:01:34.690717 (+ 18934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.214002 (+1523285us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5800855ed0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:01:36.319381 (+105379us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:36.320144 (+   763us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.413903 (+ 93759us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b1800338380. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:36.753423 (+339520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.753532 (+   109us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:36.753603 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.753728 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.753902 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169,"prepare.run_cpu_time_us":190,"prepare.run_wall_time_us":191,"raft.queue_time_us":12560,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":21971,"spinlock_wait_cycles":846964864,"thread_start_us":314724,"threads_started":1}
I20260709 13:01:37.928320 13346 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:37.926242 13347 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:37.931253 13347 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:37.930207 13346 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:37.933548 13346 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:37.933954 13346 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:37.934813 13346 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:37.984050  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:38.129719 13334 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.555s	user 0.005s	sys 0.000s
I20260709 13:01:38.136209  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1808) took 1731 ms. Trace:
I20260709 13:01:38.136682  8815 rpcz_store.cc:276] 0709 13:01:36.404325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.429373 (+ 25048us) service_pool.cc:224] Handling call
0709 13:01:36.462575 (+ 33202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.792625 (+330050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.135924 (+1343299us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540041ffd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:38.136140 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720214272}
I20260709 13:01:38.137233 13306 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:38.143345 13219 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:38.143944 13219 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.145901 13219 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.146327 13219 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:38.261940 13306 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.262588 13306 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:38.265309 13306 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:38.269601  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 239545us
W20260709 13:01:38.269939  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 350186us
I20260709 13:01:38.280386  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.281194 13288 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.281652 13026 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.282198 13026 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:38.284502  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.286096  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.287418 13264 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:38.287556 13348 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.285312 13026 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.289646  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:38.291538 13288 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:38.291923 13288 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.292265 13288 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.293519 13026 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:38.295032  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:38.296094  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.296662 13026 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:38.297278 13026 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.297708 13026 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.295307 13348 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.300866 13348 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:38.301846 13348 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:38.302781 13264 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:38.303069 13348 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.304726 13264 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:38.306501  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.311751 13264 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.324344  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.324931  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.325834 13026 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:38.328163  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.328354 13026 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:38.330230  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.333775  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.334546  8744 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:38.331959  8741 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:38.337972  9446 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:38.338063  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.338768  8741 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:38.339121  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:38.339524  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.340534  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.324935  8740 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.343611 13023 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.344079 13023 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.345736  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.345986  8745 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:01:38.346312  9441 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:38.347091  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.347653  8745 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 8.
I20260709 13:01:38.348528  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:38.348919  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.349145  9447 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.350970  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.353127 13023 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:38.353523  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.356257  8745 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:38.355638  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1814) took 1949 ms. Trace:
I20260709 13:01:38.357568  8817 rpcz_store.cc:276] 0709 13:01:36.406225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.670675 (+264450us) service_pool.cc:224] Handling call
0709 13:01:36.671116 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.680662 (+  9546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.680692 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.681129 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.756998 (+ 75869us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:36.757056 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.757153 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.355440 (+1598287us) raft_consensus.cc:1685] finished
0709 13:01:38.355462 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.355585 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9117,"raft.queue_time_us":10004,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":14761,"thread_start_us":24547,"threads_started":1,"wal-append.queue_time_us":147198,"wal-append.run_cpu_time_us":7737,"wal-append.run_wall_time_us":1393405}
I20260709 13:01:38.359922  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.356851  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.356931  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.360843  9449 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:38.361152  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.355501  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.354748  8746 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:38.362360  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.362639  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.362901  8744 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 9.
I20260709 13:01:38.363255  8746 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:38.363734  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.364106  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.364720  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.365139  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.365267  8744 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:38.378953  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:38.379758 13166 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.380311 13166 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.380671 13166 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.381563 12889 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.382037 12889 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.383507 13346 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.384102 12889 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.379325  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.386055  8744 raft_consensus.cc:2376] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13. Re-sending same reply.
I20260709 13:01:38.387066  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.391799  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.392421  8744 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.393008  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:38.366875  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:38.375783  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1804) took 2091 ms. Trace:
I20260709 13:01:38.400338  8818 rpcz_store.cc:276] 0709 13:01:36.284048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.284201 (+   153us) service_pool.cc:224] Handling call
0709 13:01:36.301729 (+ 17528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.765919 (+464190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.375543 (+1609624us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400083ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:38.375723 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901743488}
I20260709 13:01:38.377787 13248 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:01:38.366150  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.396770  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.397400  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.405882  8820 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:38.406170  8894 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.378177  8817 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.399185  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:38.407563  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1810) took 2002 ms. Trace:
I20260709 13:01:38.407944  8820 rpcz_store.cc:276] 0709 13:01:36.404991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.410756 (+  5765us) service_pool.cc:224] Handling call
0709 13:01:36.411071 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.406269 (+1995198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.407341 (+  1072us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400020dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:38.407505 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4588462336}
I20260709 13:01:38.410115 13217 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.417539  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.418570  8741 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:38.399899 12889 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.372466  9447 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:38.424592  8745 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:38.425828  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.426949 13038 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.427464 13038 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.428577 13164 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.428669 13233 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:38.429239 13233 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.429199 13164 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:38.429848 13109 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:38.430531 13109 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:38.435590  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.442034  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2102) took 1437 ms. Trace:
I20260709 13:01:38.442584  8742 rpcz_store.cc:276] 0709 13:01:37.004529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.005558 (+  1029us) service_pool.cc:224] Handling call
0709 13:01:38.441730 (+1436172us) spinlock_profiling.cc:243] Waited 70.8 ms on lock 0x7b54004632d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:38.441926 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162765184}
I20260709 13:01:38.443948  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.444886  8742 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:38.448215  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:38.449012  8817 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:38.449792  8749 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:38.454303 13240 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:38.455855  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.390388 13166 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.465719  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2153) took 1451 ms. Trace:
I20260709 13:01:38.466156  8897 rpcz_store.cc:276] 0709 13:01:37.014308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.030002 (+ 15694us) service_pool.cc:224] Handling call
0709 13:01:37.064863 (+ 34861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.169451 (+104588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.465526 (+1296075us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b5400248950. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:38.465666 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79450880}
I20260709 13:01:38.467649 13166 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:38.468204  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.300216 13288 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:38.470180 13288 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.474071  8891 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:38.477430  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.478294  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:38.478884  8820 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:38.478940 13240 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:38.480247  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.481118  8742 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:38.482241  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.527716 13233 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:38.528291  8742 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:38.529758  8742 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:38.495488  8746 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:38.419447  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.497707 13080 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.532071  8741 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:38.532146 13080 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:38.497975 13164 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.506901 12759 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:38.513418  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.706756  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.707419  8895 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:38.708191  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.708715  8895 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:38.709474  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.710453  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.516104  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:38.518649  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:38.721809  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 205857us
I20260709 13:01:38.520298  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.722698  8890 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:38.722931  8863 leader_election.cc:336] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.671s (SENT)
I20260709 13:01:38.723572  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.724228  8890 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:38.521314  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.521952  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:38.725709  8893 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.522616  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.725644 13202 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.726469  8889 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:38.727406  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.522562  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.524742  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:38.533387  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.725075  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.533522 13248 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.506330  8740 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:01:40.120028  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 743) took 1796 ms (client timeout 2129 ms). Trace:
I20260709 13:01:38.533387  8741 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:40.123099  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1838) took 1748 ms. Trace:
I20260709 13:01:40.123474  8741 rpcz_store.cc:276] 0709 13:01:38.374725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.532818 (+158093us) service_pool.cc:224] Handling call
0709 13:01:38.533085 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.122783 (+1589698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.123025 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.427436  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:38.546204  8744 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
W20260709 13:01:40.206755  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1837) took 1832 ms (client timeout 1758 ms). Trace:
I20260709 13:01:38.612953  8747 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:38.671403  8743 raft_consensus.cc:2410] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:38.696666  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:38.714627 13361 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:40.209249 13361 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:40.209666 13361 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:40.210533 13361 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.211930 13361 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.541445  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:40.207190  8744 rpcz_store.cc:269] 0709 13:01:38.374459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.391559 (+ 17100us) service_pool.cc:224] Handling call
0709 13:01:40.206672 (+1815113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.218677  8815 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
W20260709 13:01:40.219244  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2471) took 1678 ms (client timeout 1734 ms). Trace:
W20260709 13:01:40.219579  8815 rpcz_store.cc:269] 0709 13:01:38.540996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.541153 (+   157us) service_pool.cc:224] Handling call
0709 13:01:40.219186 (+1678033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.725780  8818 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:40.221038  8818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2470) took 1699 ms (client timeout 1624 ms). Trace:
W20260709 13:01:40.221401  8818 rpcz_store.cc:269] 0709 13:01:38.521563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.521711 (+   148us) service_pool.cc:224] Handling call
0709 13:01:40.220971 (+1699260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.549161  8819 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:40.224287  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2474) took 1675 ms. Trace:
I20260709 13:01:40.224630  8819 rpcz_store.cc:276] 0709 13:01:38.548541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.548687 (+   146us) service_pool.cc:224] Handling call
0709 13:01:38.548966 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.224038 (+1675072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.224234 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.552287  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:38.547580  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:40.226917  8823 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:01:40.228302  8747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2101) took 3224 ms (client timeout 2858 ms). Trace:
W20260709 13:01:40.228706  8747 rpcz_store.cc:269] 0709 13:01:37.004181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.004338 (+   157us) service_pool.cc:224] Handling call
0709 13:01:40.228098 (+3223760us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400160dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:40.228206 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2970951040}
I20260709 13:01:38.550436  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.230943  8824 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:40.231389  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2475) took 1681 ms (client timeout 2125 ms). Trace:
W20260709 13:01:40.231719  8824 rpcz_store.cc:269] 0709 13:01:38.550051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.550203 (+   152us) service_pool.cc:224] Handling call
0709 13:01:40.231335 (+1681132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.526286  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.235155 13369 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:40.235275  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.70935e+06us
W20260709 13:01:40.236636  8787 connection.cc:256] RPC call timeout handler was delayed by -1.6950367600001073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 13:01:40.236964  8787 connection.cc:256] RPC call timeout handler was delayed by -1.5960367600000609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 13:01:38.544569  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:38.602516  8862 connection.cc:256] RPC call timeout handler was delayed by -1.3888963699999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:01:38.734413  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:38.730402  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 205873us
I20260709 13:01:38.519277  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.514446  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.539893  8894 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:38.545734  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:38.602491  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:38.728134  8898 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.541335  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:38.637533  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 238487us
I20260709 13:01:38.638208  9444 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:38.639176  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.639693  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.637809  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.636843  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:38.728866  8890 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.635998  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:38.637142  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:38.728001  8889 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
W20260709 13:01:38.744501  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.739691 12759 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:38.704085 12889 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:38.505201 13038 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:38.570675 13104 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:38.653672 13080 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:38.509019 13109 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:38.715540 13164 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:38.714368 13233 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:38.498674 13254 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:38.494751 13240 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:38.578181 13305 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:38.567835 13306 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:38.717846 13260 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:39.799988  8770 debug-util.cc:398] Leaking SignalData structure 0x7b080028d660 after lost signal to thread 7638
I20260709 13:01:39.846171 13367 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:39.896837 13368 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.958995  8749 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
W20260709 13:01:40.020373  8746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.465s	user 0.003s	sys 0.021s
W20260709 13:01:40.041929  8748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.516s	user 0.015s	sys 0.031s
W20260709 13:01:40.056296  9449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.548s	user 0.022s	sys 0.014s
I20260709 13:01:40.243254  8892 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
W20260709 13:01:40.091696  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.364s	user 0.009s	sys 0.075s
W20260709 13:01:40.299605  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.572s	user 0.010s	sys 0.076s
W20260709 13:01:40.244951  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 734) took 1847 ms (client timeout 1998 ms). Trace:
W20260709 13:01:40.300139  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 1.57307e+06us
W20260709 13:01:40.300416  8894 rpcz_store.cc:269] 0709 13:01:38.397067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.397170 (+   103us) service_pool.cc:224] Handling call
0709 13:01:40.244885 (+1847715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.301405 13386 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:40.302017  8894 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:40.093302 13365 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:40.302615  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2175) took 1778 ms. Trace:
I20260709 13:01:40.302752 13365 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:40.303081  8894 rpcz_store.cc:276] 0709 13:01:38.524665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.301381 (+1776716us) service_pool.cc:224] Handling call
0709 13:01:40.301697 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.302393 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.302565 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.304554  8894 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:01:40.098448  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.021s	sys 0.029s
W20260709 13:01:40.245896  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2179) took 1720 ms (client timeout 1620 ms). Trace:
W20260709 13:01:40.305506  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.590s	user 0.022s	sys 0.030s
I20260709 13:01:40.244225  8896 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:40.305972  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 1.73819e+06us
W20260709 13:01:40.306730  8748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.781s	user 0.016s	sys 0.031s
W20260709 13:01:40.306694  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 737) took 1792 ms (client timeout 2160 ms). Trace:
I20260709 13:01:40.305078  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2176) took 1780 ms. Trace:
W20260709 13:01:40.124748  8740 rpcz_store.cc:269] 0709 13:01:38.323991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.324109 (+   118us) service_pool.cc:224] Handling call
0709 13:01:40.119956 (+1795847us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.134829  8861 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.403s	user 0.311s	sys 0.504s
I20260709 13:01:40.255749  9446 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:40.136991  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08001ee6c0 after lost signal to thread 7638
W20260709 13:01:40.248966  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 1.70818e+06us
W20260709 13:01:40.308642  8845 debug-util.cc:398] Leaking SignalData structure 0x7b0800191a80 after lost signal to thread 7641
I20260709 13:01:40.307655  8894 rpcz_store.cc:276] 0709 13:01:38.524936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.303962 (+1779026us) service_pool.cc:224] Handling call
0709 13:01:40.304242 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.304849 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.305020 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.157509  8893 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.431s	user 0.007s	sys 0.011s
W20260709 13:01:40.309298  9446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 805) took 1672 ms (client timeout 1740 ms). Trace:
W20260709 13:01:40.308290  8861 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.577s	user 0.313s	sys 0.505s
I20260709 13:01:40.309525  8893 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
W20260709 13:01:40.171691  8864 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.548s (SENT) [suppressed 46 similar messages]
W20260709 13:01:40.177673  8745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1835) took 1803 ms (client timeout 2020 ms). Trace:
W20260709 13:01:40.184185  9412 connection.cc:256] RPC call timeout handler was delayed by -1.2908107230000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:01:40.237939  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:40.268507  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:40.185175 13268 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:40.189584  8919 debug-util.cc:398] Leaking SignalData structure 0x7b080012e1a0 after lost signal to thread 7638
I20260709 13:01:40.226315  8822 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:40.342536  8919 debug-util.cc:398] Leaking SignalData structure 0x7b080017d380 after lost signal to thread 7641
W20260709 13:01:40.230089  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1815) took 3148 ms (client timeout 2737 ms). Trace:
W20260709 13:01:40.343636  8919 debug-util.cc:398] Leaking SignalData structure 0x7b08001a5d80 after lost signal to thread 8681
W20260709 13:01:40.299566  8892 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2165) took 1824 ms (client timeout 1826 ms). Trace:
W20260709 13:01:40.344259  8892 rpcz_store.cc:269] 0709 13:01:38.475412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.519028 (+ 43616us) service_pool.cc:224] Handling call
0709 13:01:40.299502 (+1780474us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.112691  9441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.021s	sys 0.064s
I20260709 13:01:38.558051  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:40.307407  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.364285  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:40.150573  8742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.619s	user 0.007s	sys 0.010s
I20260709 13:01:40.154898 13217 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:40.310999  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2473) took 1763 ms (client timeout 1950 ms). Trace:
W20260709 13:01:40.369741  8823 rpcz_store.cc:269] 0709 13:01:38.547186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.547336 (+   150us) service_pool.cc:224] Handling call
0709 13:01:40.310948 (+1763612us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.370366  8742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.838s	user 0.007s	sys 0.010s
I20260709 13:01:40.370944  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.371423  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 754) took 2011 ms. Trace:
I20260709 13:01:40.371731  8823 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:40.287899 13240 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:40.279678  9445 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:40.376008  9445 pending_rounds.cc:85] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 9
I20260709 13:01:40.376307  9445 pending_rounds.cc:107] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:01:40.376609  9445 raft_consensus.cc:2889] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:40.311935  8845 debug-util.cc:398] Leaking SignalData structure 0x7b0800062ea0 after lost signal to thread 8681
W20260709 13:01:40.379187  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08000cde40 after lost signal to thread 8773
W20260709 13:01:40.379570  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08001f22e0 after lost signal to thread 8774
W20260709 13:01:40.381009  8845 debug-util.cc:398] Leaking SignalData structure 0x7b0800394320 after lost signal to thread 8848
I20260709 13:01:40.263131  9448 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:40.312069  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:40.310097  8893 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2167) took 1789 ms (client timeout 1547 ms). Trace:
W20260709 13:01:40.382304  8893 rpcz_store.cc:269] 0709 13:01:38.520931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.521059 (+   128us) service_pool.cc:224] Handling call
0709 13:01:40.310032 (+1788973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.262104  9447 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.263969  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 744) took 1714 ms. Trace:
W20260709 13:01:40.295222  8770 debug-util.cc:398] Leaking SignalData structure 0x7b080028daa0 after lost signal to thread 7641
I20260709 13:01:40.297896  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1833) took 1924 ms. Trace:
I20260709 13:01:40.246656  8891 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:40.308821  9449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.800s	user 0.024s	sys 0.016s
I20260709 13:01:40.386042  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 869) took 1982 ms. Trace:
I20260709 13:01:40.279265 12889 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:40.313927  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 142273us
I20260709 13:01:40.309469  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:40.290511  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.239176  8787 connection.cc:256] RPC call timeout handler was delayed by -1.5087993969999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:01:40.390003  8787 connection.cc:256] RPC call timeout handler was delayed by -1.4124367600001051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 13:01:40.390129  8789 connection.cc:256] RPC call timeout handler was delayed by -1.5445969400000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:01:40.390748  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.391124  8789 connection.cc:256] RPC call timeout handler was delayed by -1.3221300050000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
W20260709 13:01:40.391281  8787 connection.cc:256] RPC call timeout handler was delayed by -1.3782367600000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
W20260709 13:01:40.391654  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2276300050000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:01:40.391860  8787 connection.cc:256] RPC call timeout handler was delayed by -1.2441367600000603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:01:40.392174  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2150300050000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:01:40.392354  8787 connection.cc:256] RPC call timeout handler was delayed by -1.2171367600001304s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 13:01:40.392668  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0012193330001082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:01:40.305738  8897 rpcz_store.cc:269] 0709 13:01:38.525774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.545487 (+ 19713us) service_pool.cc:224] Handling call
0709 13:01:40.245828 (+1700341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.393720  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.394124  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 741) took 1846 ms. Trace:
I20260709 13:01:40.394548  8897 rpcz_store.cc:276] 0709 13:01:38.548015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.393360 (+1845345us) service_pool.cc:224] Handling call
0709 13:01:40.393644 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.393817 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.393840 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.393887 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:40.393918 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.393969 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.394074 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.307183  8896 rpcz_store.cc:269] 0709 13:01:38.514106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.514219 (+   113us) service_pool.cc:224] Handling call
0709 13:01:40.306632 (+1792413us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.309954  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.402232  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.295042 13260 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:40.412434  9448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 909) took 1775 ms (client timeout 1527 ms). Trace:
W20260709 13:01:40.412779  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103429us
I20260709 13:01:40.291566  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:40.414079  8893 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:01:40.414261  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122828us
I20260709 13:01:40.414711  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 740) took 1867 ms. Trace:
I20260709 13:01:40.414920  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.415058  8893 rpcz_store.cc:276] 0709 13:01:38.547703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.388807 (+1841104us) service_pool.cc:224] Handling call
0709 13:01:40.389138 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.414470 (+ 25332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.414660 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.396186  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 743) took 1847 ms. Trace:
I20260709 13:01:40.254648  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 910) took 1617 ms. Trace:
I20260709 13:01:40.341190  8894 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:40.416575  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2177) took 1891 ms. Trace:
I20260709 13:01:40.417053  8894 rpcz_store.cc:276] 0709 13:01:38.525198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.309515 (+1784317us) service_pool.cc:224] Handling call
0709 13:01:40.309768 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.416217 (+106449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.416494 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.344944  8919 debug-util.cc:398] Leaking SignalData structure 0x7b0800134680 after lost signal to thread 8773
W20260709 13:01:40.418197  8919 debug-util.cc:398] Leaking SignalData structure 0x7b08001af520 after lost signal to thread 8774
W20260709 13:01:40.419700  8919 debug-util.cc:398] Leaking SignalData structure 0x7b0800368ee0 after lost signal to thread 8848
W20260709 13:01:40.420029  8919 debug-util.cc:398] Leaking SignalData structure 0x7b080010b9a0 after lost signal to thread 8849
W20260709 13:01:40.412909  9448 rpcz_store.cc:269] 0709 13:01:38.636776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.636910 (+   134us) service_pool.cc:224] Handling call
0709 13:01:40.412356 (+1775446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.061446 13248 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:40.345124  9441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.618s	user 0.022s	sys 0.065s
W20260709 13:01:40.310652  8745 rpcz_store.cc:269] 0709 13:01:38.373888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.427153 (+ 53265us) service_pool.cc:224] Handling call
0709 13:01:40.177598 (+1750445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.416049  9443 rpcz_store.cc:276] 0709 13:01:38.637073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.637598 (+   525us) service_pool.cc:224] Handling call
0709 13:01:40.254575 (+1616977us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.309705  9446 rpcz_store.cc:269] 0709 13:01:38.636494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.636597 (+   103us) service_pool.cc:224] Handling call
0709 13:01:40.309225 (+1672628us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.343011  8743 rpcz_store.cc:269] 0709 13:01:37.081260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.366655 (+1285395us) service_pool.cc:224] Handling call
0709 13:01:40.229904 (+1863249us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540013bfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:40.230026 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636907776}
W20260709 13:01:40.350080  8816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2472) took 1806 ms (client timeout 1893 ms). Trace:
W20260709 13:01:40.424120  8816 rpcz_store.cc:269] 0709 13:01:38.543891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.544300 (+   409us) service_pool.cc:224] Handling call
0709 13:01:40.349852 (+1805552us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54001b8854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:40.350000 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69555072}
I20260709 13:01:40.288897  8898 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:40.356422  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 115858us
W20260709 13:01:40.425161  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2169) took 1903 ms (client timeout 1556 ms). Trace:
I20260709 13:01:40.357632 13390 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.357834  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 740) took 2043 ms. Trace:
I20260709 13:01:40.361161  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.415953  8897 rpcz_store.cc:276] 0709 13:01:38.548750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.395441 (+1846691us) service_pool.cc:224] Handling call
0709 13:01:40.395694 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.395890 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.395914 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.395953 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:40.395983 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.396043 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.396136 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.362283  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.362394  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.251739  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 912) took 1614 ms. Trace:
W20260709 13:01:40.426915  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 125438us
I20260709 13:01:40.426033  8748 rpcz_store.cc:276] 0709 13:01:38.314296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.314403 (+   107us) service_pool.cc:224] Handling call
0709 13:01:38.314666 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.314851 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:38.525744 (+210893us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.357403 (+1831659us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.357461 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:40.357512 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.357596 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.357777 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":210876,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"thread_start_us":2041792,"threads_started":3,"wal-append.queue_time_us":75709,"wal-append.run_cpu_time_us":1153,"wal-append.run_wall_time_us":1001168}
I20260709 13:01:40.241860 13369 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.286139 13233 consensus_queue.cc:1243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:40.286983 13254 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:40.297973 13306 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:40.288684 13305 consensus_queue.cc:1243] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:40.367616  8818 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:40.368997  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 739) took 1821 ms. Trace:
I20260709 13:01:40.447867  8817 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:40.448087  8891 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 11 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 12.
I20260709 13:01:40.373214  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:40.268752  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3047376250000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3028s for a response
I20260709 13:01:40.265136  8644 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 10 to 13, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 13 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.294435 13038 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:40.251892 13075 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:40.384959  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8300 after lost signal to thread 8681
I20260709 13:01:40.253499  9442 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:40.384665  8749 rpcz_store.cc:276] 0709 13:01:38.373274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.469150 (+ 95876us) service_pool.cc:224] Handling call
0709 13:01:40.297678 (+1828528us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54004646d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:40.297825 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3406472064}
I20260709 13:01:40.384198  8889 rpcz_store.cc:276] 0709 13:01:38.549065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.727167 (+178102us) service_pool.cc:224] Handling call
0709 13:01:40.263901 (+1536734us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.381845  8820 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:40.258800 13372 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:40.452836  9442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 913) took 1815 ms (client timeout 1877 ms). Trace:
W20260709 13:01:40.506263  8647 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 301) took 26449 ms (client timeout 15000 ms). Trace:
I20260709 13:01:40.422878 13361 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:40.386487  9449 rpcz_store.cc:276] 0709 13:01:38.403405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.405099 (+  1694us) service_pool.cc:224] Handling call
0709 13:01:38.405447 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.405831 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:38.405859 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.508321 (+102462us) log.cc:844] Serialized 52 byte log entry
0709 13:01:40.385539 (+1877218us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:40.385608 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.385704 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.385770 (+    66us) raft_consensus.cc:1685] finished
0709 13:01:40.385788 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.385968 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4102,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":1876746,"threads_started":1,"wal-append.queue_time_us":22577,"wal-append.run_cpu_time_us":11966,"wal-append.run_wall_time_us":1682848}
I20260709 13:01:40.341344 13365 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:40.389535  8864 connection.cc:256] RPC call timeout handler was delayed by -1.3645385559999568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:01:40.513151  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.513653  8864 connection.cc:256] RPC call timeout handler was delayed by -1.2070385559999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
W20260709 13:01:40.513994  8864 connection.cc:256] RPC call timeout handler was delayed by -1.1600997720000805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 13:01:40.298926  8746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.743s	user 0.004s	sys 0.022s
I20260709 13:01:40.451107 13038 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:40.517443 13361 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.517987  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1842) took 2141 ms. Trace:
I20260709 13:01:40.518260 13038 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.519341 13038 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.518369  8746 rpcz_store.cc:276] 0709 13:01:38.376910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.554082 (+177172us) service_pool.cc:224] Handling call
0709 13:01:38.554398 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.554754 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.554780 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.555198 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.517627 (+1962429us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:40.517669 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.517746 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.517790 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:40.517802 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.517926 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":1961996,"threads_started":1,"wal-append.queue_time_us":26275,"wal-append.run_cpu_time_us":1963,"wal-append.run_wall_time_us":1001970}
I20260709 13:01:40.519682 13361 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:40.403710  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.345531  8890 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:40.424909  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:40.517174  8864 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 2.450s (SENT)
I20260709 13:01:40.427523  8824 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:01:40.426033  8919 debug-util.cc:398] Leaking SignalData structure 0x7b0800264440 after lost signal to thread 8922
W20260709 13:01:40.521209  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2170) took 1997 ms (client timeout 2192 ms). Trace:
W20260709 13:01:40.521605  8890 rpcz_store.cc:269] 0709 13:01:38.523361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.724821 (+201460us) service_pool.cc:224] Handling call
0709 13:01:40.521148 (+1796327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.426247  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:40.522147  8919 debug-util.cc:398] Leaking SignalData structure 0x7b080041c0e0 after lost signal to thread 9473
W20260709 13:01:40.522501  8919 debug-util.cc:398] Leaking SignalData structure 0x7b0800423d20 after lost signal to thread 9474
I20260709 13:01:40.427677  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:40.522594  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 96434.4us
W20260709 13:01:40.425567  8898 rpcz_store.cc:269] 0709 13:01:38.522214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.522322 (+   108us) service_pool.cc:224] Handling call
0709 13:01:40.425095 (+1902773us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.431252  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 1.73496e+06us
I20260709 13:01:40.431654  8815 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:40.524205  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.285205 13194 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:01:40.386565 13217 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.434166  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 871) took 1798 ms. Trace:
I20260709 13:01:40.520570  8644 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.550045  9445 rpcz_store.cc:276] 0709 13:01:38.635836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.635957 (+   121us) service_pool.cc:224] Handling call
0709 13:01:38.636222 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.377448 (+1741226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.426138 (+ 48690us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400740254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:40.434084 (+  7946us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3779598080}
I20260709 13:01:40.435034 13375 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:40.435231 13379 raft_consensus.cc:2764] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:40.253981 13261 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:01:40.438305  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 865) took 2091 ms. Trace:
I20260709 13:01:40.375780 13240 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:40.446990  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 98856.5us
I20260709 13:01:40.518375 13080 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:40.386360 13377 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:40.295804 13368 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:40.451629  9448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 914) took 1813 ms (client timeout 1758 ms). Trace:
W20260709 13:01:40.573611  9448 rpcz_store.cc:269] 0709 13:01:38.638241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.451459 (+1813218us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:40.451571 (+   112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:40.360034 13268 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.451601  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 738) took 1904 ms (client timeout 1592 ms). Trace:
W20260709 13:01:40.574590  8891 rpcz_store.cc:269] 0709 13:01:38.546982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.602272 (+ 55290us) service_pool.cc:224] Handling call
0709 13:01:40.451532 (+1849260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.422482 13383 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.430895 13248 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.371994  8742 rpcz_store.cc:276] 0709 13:01:38.359558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.530585 (+171027us) service_pool.cc:224] Handling call
0709 13:01:38.530881 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.531253 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.531286 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.531723 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.370950 (+1839227us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:40.371011 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.371103 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.371185 (+    82us) raft_consensus.cc:1685] finished
0709 13:01:40.371205 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.371359 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"thread_start_us":1838830,"threads_started":1,"wal-append.queue_time_us":59364,"wal-append.run_cpu_time_us":2422,"wal-append.run_wall_time_us":1002606}
I20260709 13:01:40.431808 13374 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:40.453837  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6ba0 after lost signal to thread 8773
W20260709 13:01:40.507104  9442 rpcz_store.cc:269] 0709 13:01:38.637929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.639522 (+  1593us) service_pool.cc:224] Handling call
0709 13:01:40.452780 (+1813258us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:40.577402  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003c94c0 after lost signal to thread 8774
W20260709 13:01:40.387866  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08003198a0 after lost signal to thread 8849
W20260709 13:01:40.578590  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8ce0 after lost signal to thread 8848
W20260709 13:01:40.578847  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08002da9e0 after lost signal to thread 8849
W20260709 13:01:40.579384  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08000df7e0 after lost signal to thread 8922
W20260709 13:01:40.579855  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003c5b80 after lost signal to thread 8922
W20260709 13:01:40.580116  8770 debug-util.cc:398] Leaking SignalData structure 0x7b0800037fc0 after lost signal to thread 8923
W20260709 13:01:40.580601  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08000497e0 after lost signal to thread 9473
W20260709 13:01:40.580864  8845 debug-util.cc:398] Leaking SignalData structure 0x7b0800310580 after lost signal to thread 9474
I20260709 13:01:40.451038 13202 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:01:40.446188 13306 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:40.440217 13369 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.511919  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 917) took 1872 ms (client timeout 1769 ms). Trace:
W20260709 13:01:40.585536  9449 rpcz_store.cc:269] 0709 13:01:38.639052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.511735 (+1872683us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:40.511858 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:40.512115 13394 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.366072  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:40.440218 13390 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:40.587301  8821 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.446195 12889 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:40.429957  9447 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:40.427203  9444 rpcz_store.cc:276] 0709 13:01:38.637638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.637773 (+   135us) service_pool.cc:224] Handling call
0709 13:01:38.638037 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.251495 (+1613458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.251683 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.533461  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:40.588673  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.589228  9443 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:40.539692  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.540952  8822 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
W20260709 13:01:40.590651  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2476) took 2038 ms (client timeout 2007 ms). Trace:
I20260709 13:01:40.562362  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.562664 13399 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:40.592231  8822 rpcz_store.cc:269] 0709 13:01:38.551835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.552032 (+   197us) service_pool.cc:224] Handling call
0709 13:01:40.590587 (+2038555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.564029 13398 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:40.567545  8890 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:40.447988  8892 rpcz_store.cc:276] 0709 13:01:38.547355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.344918 (+1797563us) service_pool.cc:224] Handling call
0709 13:01:40.368313 (+ 23395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.368562 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.368600 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.368659 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:40.368708 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.368780 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.368926 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.510023 13254 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:40.572050  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.568266  9441 rpcz_store.cc:276] 0709 13:01:38.347320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.349968 (+  2648us) service_pool.cc:224] Handling call
0709 13:01:38.350225 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.350408 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:38.726591 (+376183us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.437987 (+1711396us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.438035 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:40.438083 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.438142 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.438249 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1557,"prepare.queue_time_us":239677,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":16065,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":2085272,"threads_started":3,"wal-append.queue_time_us":1077279}
I20260709 13:01:40.594525  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:40.581192  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003eb080 after lost signal to thread 9473
I20260709 13:01:40.552575 13075 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:40.583130  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 158304us
I20260709 13:01:40.601890  8897 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:40.590301  8644 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.591289  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:40.601900  8712 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.979s (SENT)
I20260709 13:01:40.592500  8898 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:40.563712  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 147985us
I20260709 13:01:40.603261  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:40.595186  8770 debug-util.cc:398] Leaking SignalData structure 0x7b0800213360 after lost signal to thread 9474
I20260709 13:01:40.594617  8889 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:01:40.604722  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003da3c0 after lost signal to thread 13038
W20260709 13:01:40.591250  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 51704.7us
W20260709 13:01:40.605041  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08002d42e0 after lost signal to thread 13109
W20260709 13:01:40.596634  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 870) took 1961 ms (client timeout 1678 ms). Trace:
I20260709 13:01:40.605628  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.583847 13374 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.595194  8894 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:40.609438  8770 debug-util.cc:398] Leaking SignalData structure 0x7b080037cce0 after lost signal to thread 13240
W20260709 13:01:40.581471  8845 debug-util.cc:398] Leaking SignalData structure 0x7b08003b9840 after lost signal to thread 13121
W20260709 13:01:40.597802  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 915) took 1959 ms (client timeout 2233 ms). Trace:
W20260709 13:01:40.610134  9443 rpcz_store.cc:269] 0709 13:01:38.638548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.533091 (+1894543us) service_pool.cc:224] Handling call
0709 13:01:40.597749 (+ 64658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.599112 13402 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.454906 13305 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:40.599164  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 2.04152e+06us
I20260709 13:01:40.600841 13404 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:40.596728  9440 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:40.573159 13080 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:40.624778  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:40.606494  8893 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 8.
I20260709 13:01:40.604907  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.593581 13379 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.602504 13365 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:40.572667 13370 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:40.608421  8789 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.559s (SENT)
I20260709 13:01:40.608973  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:40.609774  8770 debug-util.cc:398] Leaking SignalData structure 0x7b0800252ce0 after lost signal to thread 13254
W20260709 13:01:40.671563  8770 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6da0 after lost signal to thread 13344
I20260709 13:01:40.623294  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.672279  8898 raft_consensus.cc:2410] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:40.671021  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.621680  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:40.658504  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.643322  8894 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:40.606489  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.625986  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.634367  8644 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 7 to 9, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:01:40.607913  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:40.601755 13075 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:40.678025  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:40.526494 13260 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.507249  8647 rpcz_store.cc:269] 0709 13:01:14.056603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.056848 (+   245us) service_pool.cc:224] Handling call
0709 13:01:14.346974 (+290126us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:01:40.506166 (+26159192us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:34.829972 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:34.830642 (+   670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:34.830687 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:01:34.830706 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:34.832341 (+  1635us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:34.832427 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:34.832455 (+    28us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:01:35.094308 (+261853us) tablet.cc:701] Row locks acquired
0709 13:01:35.108268 (+ 13960us) write_op.cc:260] PREPARE: finished
0709 13:01:35.113499 (+  5231us) write_op.cc:270] Start()
0709 13:01:35.135565 (+ 22066us) write_op.cc:276] Timestamp: P: 1783602095113419 usec, L: 0
0709 13:01:35.158296 (+ 22731us) op_driver.cc:348] REPLICATION: starting
0709 13:01:35.434233 (+275937us) log.cc:844] Serialized 7536 byte log entry
0709 13:01:36.873843 (+1439610us) op_driver.cc:464] REPLICATION: finished
0709 13:01:37.254616 (+380773us) write_op.cc:301] APPLY: starting
0709 13:01:37.254744 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:37.552027 (+297283us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:37.552138 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:37.629823 (+ 77685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:37.630084 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:01:37.630115 (+    31us) write_op.cc:312] APPLY: finished
0709 13:01:37.630480 (+   365us) log.cc:844] Serialized 172 byte log entry
0709 13:01:37.630688 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:37.630877 (+   189us) write_op.cc:454] Released schema lock
0709 13:01:37.630972 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:37.631393 (+   421us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:37.631416 (+    23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":230545,"reported_tablets":49,"spinlock_wait_cycles":385511168,"child_traces":[["op",{"apply.queue_time_us":351970,"apply.run_cpu_time_us":31407,"apply.run_wall_time_us":376794,"cfile_cache_hit":74,"cfile_cache_hit_bytes":77293,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6216,"cfile_init":2,"lbm_read_time_us":19647,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":10,"num_ops":19,"prepare.queue_time_us":188041,"prepare.run_cpu_time_us":32789,"prepare.run_wall_time_us":845201,"raft.queue_time_us":425393,"raft.run_cpu_time_us":22215,"raft.run_wall_time_us":410988,"replication_time_us":1615662,"thread_start_us":695474,"threads_started":4,"wal-append.queue_time_us":620074,"wal-append.run_cpu_time_us":69561,"wal-append.run_wall_time_us":2498920}]]}
I20260709 13:01:40.694429  8893 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.690829  8897 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
W20260709 13:01:40.632022  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 204441us
I20260709 13:01:40.679327  8895 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:40.679579  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 52189.8us
I20260709 13:01:40.697978  8895 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:40.698318  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:40.586045 13377 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.647008  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.679000  8822 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:40.680884  8889 raft_consensus.cc:2410] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:01:40.605655  9447 rpcz_store.cc:269] 0709 13:01:38.635523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.635683 (+   160us) service_pool.cc:224] Handling call
0709 13:01:40.596566 (+1960883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.670337 13399 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.681494  8894 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:40.618817 13374 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.683112  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:40.683975  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 75602us
I20260709 13:01:40.624437 13402 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:40.698720  8895 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:40.725154  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:40.725865  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 119530us
I20260709 13:01:40.725039  8823 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:40.586324 13383 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.608906 13254 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.777411  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.677784 13109 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:40.788269 13403 consensus_queue.cc:1175] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7fd110c312c14e17a2bbe6d453493cf2" responder_term: 11 status { last_received { term: 6 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:40.790628 13109 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:40.793143  9446 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:01:40.794857 13109 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.796103 13402 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:40.796792 13109 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.731148 13397 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:01:40.781132  8889 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:40.625038 13080 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.800273  8895 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.791405  8890 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:40.677820 13365 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.618284 13393 raft_consensus.cc:2764] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:40.798574 13401 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:01:40.822160 13268 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:40.822770 13268 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:40.824072 13268 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:40.824525 13268 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:40.797013 13407 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:40.797856  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 176316us
I20260709 13:01:40.799185  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:40.701884  8898 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:01:40.849355  8898 pending_rounds.cc:85] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 10
I20260709 13:01:40.849733  8898 pending_rounds.cc:107] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:01:40.850037  8898 raft_consensus.cc:2889] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.814005  8644 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.790915  8891 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:40.798759  8789 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.770s (SENT)
I20260709 13:01:40.851833  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.788942  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:40.817660 13217 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:40.625964 13305 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.817377 13233 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:40.819895 13260 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.821182  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 916) took 2182 ms. Trace:
W20260709 13:01:40.826122  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 224286us
I20260709 13:01:40.670226 13379 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.700558 13075 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.826133 13365 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:40.875413 13379 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:40.702140  8897 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:40.791380 13254 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:40.726796 13376 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:40.677271 13306 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:40.854712  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 55988.9us
I20260709 13:01:40.726649 13361 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:40.857043  9446 rpcz_store.cc:276] 0709 13:01:38.638793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.535910 (+1897117us) service_pool.cc:224] Handling call
0709 13:01:40.571344 (+ 35434us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:40.794522 (+223178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.820888 (+ 26366us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54006dd654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:40.821118 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508053248}
I20260709 13:01:40.879061 13382 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:40.828356 13368 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:40.879522 13382 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:40.880038  8789 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.809s (SENT)
I20260709 13:01:40.880820 13377 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:40.817494 13397 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:01:40.859061 13268 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.860302 13369 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:40.860707 13217 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:40.682257 13370 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:40.865845  8891 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:40.866074  8895 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:40.861168  8644 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:40.926617  8896 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:40.926679  8895 pending_rounds.cc:85] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 5
I20260709 13:01:40.928498  8895 pending_rounds.cc:107] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:01:40.928838  8895 raft_consensus.cc:2889] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.871842 12889 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:40.873044  8712 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.163s (SENT)
I20260709 13:01:40.929795  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:40.880422 13382 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:01:40.881402  8789 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.046s (SENT)
I20260709 13:01:40.933341  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.934903  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2174) took 2410 ms. Trace:
I20260709 13:01:40.748234 13406 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:40.900197  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 327972us
I20260709 13:01:40.937057  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:40.902566  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44468.2us
W20260709 13:01:40.907470  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 118653us
I20260709 13:01:40.918290  8821 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:40.939778  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:40.940814  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.864069 13248 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.931123  9440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 911) took 2287 ms (client timeout 1734 ms). Trace:
W20260709 13:01:40.941148  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 241354us
I20260709 13:01:40.942384  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:40.941519  9440 rpcz_store.cc:269] 0709 13:01:38.637358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.638921 (+  1563us) service_pool.cc:224] Handling call
0709 13:01:40.924080 (+2285159us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54004a4554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:40.924231 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791463808}
I20260709 13:01:40.901947 12759 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:40.888315 13368 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:40.932392  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 155454us
I20260709 13:01:40.628302 13404 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:40.935425  8895 rpcz_store.cc:276] 0709 13:01:38.524359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.733629 (+209270us) service_pool.cc:224] Handling call
0709 13:01:38.734826 (+  1197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.679235 (+1944409us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b5400337820. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d11c0 00007f90f64d2231 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:01:40.934557 (+255322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.934829 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4256649984}
I20260709 13:01:40.952447  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:40.952696  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71336.7us
I20260709 13:01:40.749449 13399 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:40.950977  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 77977.8us
I20260709 13:01:40.953299  9449 raft_consensus.cc:2368] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
W20260709 13:01:40.959483  8712 leader_election.cc:336] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.193s (SENT)
I20260709 13:01:40.954700  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:40.960072  9447 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:40.961385  9449 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:01:40.962503  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 358281us
W20260709 13:01:40.962670  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.968979  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:40.969729  9440 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:40.970296  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:40.968995  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:40.925616 13361 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:40.931882 12889 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:40.921758 13408 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:40.978435  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 353790us
I20260709 13:01:40.979050  8644 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 5 to 11, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 11 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:40.973227  8789 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.013s (SENT)
W20260709 13:01:40.974463  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 301121us
I20260709 13:01:40.902184 13377 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:01:40.975754  9446 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:40.981127  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.886997 13395 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.155s
W20260709 13:01:40.982822  8789 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.051s (SENT)
W20260709 13:01:40.984282  8789 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.230s (SENT)
I20260709 13:01:40.984786  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:40.985630  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:40.986425  8823 raft_consensus.cc:2376] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12. Re-sending same reply.
I20260709 13:01:40.989236  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.990406  8819 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:01:40.991329  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.992112  8823 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:40.956007 13368 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:41.002967  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.003641  8822 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
W20260709 13:01:41.005049  8789 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.563s (SENT)
I20260709 13:01:40.972255 13382 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.005671  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.007351  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.011162  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.011312  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:40.969225 13376 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.008328 13373 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.009047  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:41.002851 12759 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:40.972548 13379 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.010514  9440 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:41.008008 13408 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:40.926045 13407 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:01:40.971215 13372 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.024870  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:41.026229 13104 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:01:41.036527  9447 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.13-11.13]
I20260709 13:01:41.046294  9446 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.047874  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.054816  9440 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:41.053635  8644 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 5 to 8, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 8 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:41.058176  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.029886  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.016863 13368 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:41.050086  8821 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:41.002627 13217 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:01:41.027204 13386 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:41.024006 13260 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:41.063180  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 105936us
I20260709 13:01:41.056119  8644 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 6 to 9, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 9 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:41.074301  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.009150 13075 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.030328 13419 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:41.068260 13410 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:01:41.036612 13415 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.008031 13361 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.010061 13416 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:41.073349  8644 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 6 to 11, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 11 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:01:41.072397  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82937.5us
W20260709 13:01:41.076615  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 117937us
W20260709 13:01:41.050412  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.982185 13240 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 11, Last appended: 11.13, Last appended by leader: 13, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:41.106357  8863 connection.cc:256] RPC call timeout handler was delayed by -1.8962935640000751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:01:41.109601  8863 connection.cc:256] RPC call timeout handler was delayed by -1.8575935640000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 13:01:41.109931  8863 connection.cc:256] RPC call timeout handler was delayed by -1.6997016899999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 13:01:41.110203  8863 connection.cc:256] RPC call timeout handler was delayed by -1.562901689999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 13:01:41.110478  8863 connection.cc:256] RPC call timeout handler was delayed by -1.4882016899999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
W20260709 13:01:41.110740  8863 connection.cc:256] RPC call timeout handler was delayed by -1.4639016899999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
W20260709 13:01:41.110998  8863 connection.cc:256] RPC call timeout handler was delayed by -1.3487016899998707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 13:01:41.111255  8863 connection.cc:256] RPC call timeout handler was delayed by -1.1975016899999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2316s for a response
W20260709 13:01:41.112154  8863 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.527s (SENT)
W20260709 13:01:41.112690  8863 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.570s (SENT)
I20260709 13:01:41.030915 13390 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.107048  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.113022  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.113942  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 320642us
I20260709 13:01:41.066658 13104 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.116381  9446 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:41.117127  8788 leader_election.cc:400] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:41.123539  8788 leader_election.cc:403] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:41.123700  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.125458  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:41.128957  8863 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.822s (SENT)
I20260709 13:01:41.129948  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.134088  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:41.136467 13202 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.137492 13202 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:41.138881  8863 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.877s (SENT)
I20260709 13:01:41.139986  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:41.099910 13419 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.139101  8788 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:41.104058 13394 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.148787  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.199987  8821 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:41.127633  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:41.127825 13395 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.199293 13419 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.183377 13305 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.104485 13415 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.204641 13415 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.122933 13386 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:01:41.205736  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.206104 13422 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:41.206526 13422 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.206857 13422 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.207609 13423 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:41.141964 13240 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.208374 13423 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.209120 13423 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.209465 13423 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.209802 13423 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.097687 13408 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.210548 13408 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.212663 13408 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:41.140549  8863 leader_election.cc:336] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.910s (SENT)
W20260709 13:01:41.214798  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 74310.7us
W20260709 13:01:41.215583  8863 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.935s (SENT)
I20260709 13:01:41.216017  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.200510  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61528us
I20260709 13:01:41.219954 13109 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:41.220758 13383 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.221453 13394 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.221957 13394 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:41.222963 13394 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.160512 12889 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:41.224996 13396 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.371s	user 0.001s	sys 0.004s
I20260709 13:01:41.225499  9449 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:41.246186  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 118750us
I20260709 13:01:41.226680 13194 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:41.226740 13399 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.228082 13075 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.231357 13365 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.233033 13404 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.226819 13261 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:41.236757 13306 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.192996 13372 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.143249 13414 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.249356  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.142483 13407 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:41.161542 13430 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.251261  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:41.257483  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.320466  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.259392  8863 leader_election.cc:336] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.993s (SENT)
W20260709 13:01:41.260393  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 186230us
W20260709 13:01:41.265753  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125867us
I20260709 13:01:41.265733  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:41.294999  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.321991  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.295526  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.192533 13413 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.141618 13260 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.302021 12889 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:41.302479  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 337892us
W20260709 13:01:41.336024  8712 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.776s (SENT)
I20260709 13:01:41.336545  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.303031 13404 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.303858 13394 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.310593 13370 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.312187 13377 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.271090 13397 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:41.287130 13261 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:41.321069  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.322953  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.321776  9446 raft_consensus.cc:2376] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13. Re-sending same reply.
I20260709 13:01:41.323050  8824 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:41.281744 13305 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.315841 13392 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.337694 13374 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.509s	user 0.003s	sys 0.006s
I20260709 13:01:41.148471 13390 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.352509  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.315395 13406 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.354482  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.360455  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.360532  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.412405  8740 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.412596  8895 raft_consensus.cc:2376] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15. Re-sending same reply.
I20260709 13:01:41.413342  8896 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 11 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 12.
W20260709 13:01:41.360705  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 254873us
I20260709 13:01:41.361301  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.331166 13407 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.369974  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.417658  8891 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:41.364591  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.365257  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:41.365657  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 106355us
I20260709 13:01:41.419252  8893 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
W20260709 13:01:41.419396  8863 leader_election.cc:336] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.020s (SENT)
I20260709 13:01:41.419834  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.366460  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 43613.4us
I20260709 13:01:41.333230 13413 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.368544  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.368887  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.425563  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.426043  8898 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:41.338464 13260 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:41.410840  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:41.349609 13397 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:41.420915  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.148s (SENT)
I20260709 13:01:41.432700  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.433876  8863 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.224s (SENT)
I20260709 13:01:41.434273  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:41.435287  8863 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.316s (SENT)
I20260709 13:01:41.429966 13260 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:41.436054  8863 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.450s (SENT)
I20260709 13:01:41.437222  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.422237  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.370400  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.438165  8743 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:41.426841  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.425354 13390 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.429111 13428 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.430222  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.363541 13305 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:41.441713  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.436403  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.442225 13413 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.442497 13442 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.442525  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.410640 13416 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.450676  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.410940 13399 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:41.452189 13389 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:01:41.454242  8863 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.737s (SENT)
I20260709 13:01:41.461416  8889 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:41.469766  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.420918 13376 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:41.345144 13404 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.451570 13417 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:41.461462  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.337991 13430 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.472103  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.467440  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.477761  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.438627 13397 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.484210  8740 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:41.424314  8742 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12. Re-sending same reply.
I20260709 13:01:41.355912 13377 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.488494  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:41.437098 13401 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:41.452832 13368 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.457506 13367 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.460633 13305 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:41.498091  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.506469  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52299us
I20260709 13:01:41.434238 13386 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:01:41.468762 13104 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:41.457650 13420 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.576323 13382 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.369021 13392 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:41.577042  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 159778us
I20260709 13:01:41.643362  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.581714 13416 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.583315 13390 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:41.587049  8863 leader_election.cc:336] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 3.199s (SENT)
I20260709 13:01:41.647548  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.434028 13406 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.410882 13374 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.513943 13417 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:41.502568  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:41.621492  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.502610  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.370641 13194 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.633463 13217 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.343555 13414 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:01:41.631235  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.631937  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.485800 13413 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.481331 12889 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.638399  8890 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:41.503480 13446 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.507560 13432 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:41.514032 13075 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.507215  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.645989  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:41.705152  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59252.9us
I20260709 13:01:41.649785 13376 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.590394 13409 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:41.662343  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 75361.1us
I20260709 13:01:41.508589  8819 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:41.589221 13419 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:41.668730  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 166175us
I20260709 13:01:41.481657 13080 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.652786 13260 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.479053 13261 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.653515 13428 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:41.714984 13376 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.726999 13260 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.678978 13217 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.711594 13365 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:41.680495  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48721.9us
I20260709 13:01:41.730193  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.642283 13423 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.688047 13395 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:41.715646 13409 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.684052 13446 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.696635 13414 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.698266 13368 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.711216 13379 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.671643 13389 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.514266 13407 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:41.707278  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 200228us
I20260709 13:01:41.709621  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:41.671010 13393 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:41.671479 13404 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:41.665805  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 163352us
W20260709 13:01:41.713276  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.742836  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.728820 13376 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.715183 13306 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.730657 13397 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.734655 13075 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:41.693754  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:41.756531  8815 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.731658 13409 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.700408 13305 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.742908  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:41.718647  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:41.732470 13446 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.747267  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:41.767042  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.720111 13429 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:41.767702 13448 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:41.767894 13429 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:41.768349 13448 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:41.768402 13429 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.681593  9447 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:41.769325 13429 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:41.684092 13417 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.733238 13268 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.778724  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:41.753547 13406 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.754091 13397 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.754326 13392 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.736855 13379 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.767169 13419 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:41.748927  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.811663  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.777964  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.813640  8746 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:41.772329  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53826.1us
I20260709 13:01:41.723704 13248 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:41.772550  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.777856  8743 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:41.817221  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.749586 13414 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:41.797914  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66354.9us
I20260709 13:01:41.799029  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.789821 13404 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:41.822867  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:41.823544  8824 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.790163 13407 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:41.794409 13426 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.806838  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:41.808363  8712 leader_election.cc:336] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 3.117s (SENT)
I20260709 13:01:41.826992  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.809330  8819 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.809590  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:41.828032  8823 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.808368  8894 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:41.761754 13423 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:41.814842  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.818894  8740 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:41.814759 13379 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.815899 13389 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.788828 13306 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:41.806141  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.821959  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.880375  8898 raft_consensus.cc:2376] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9. Re-sending same reply.
I20260709 13:01:41.824259  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.756349 13075 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:41.826516 13426 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.829411  8742 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:41.834398  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.817243  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.761765 13393 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:41.863137  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.810513 13438 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.875540  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.931217  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.931800  8747 raft_consensus.cc:2368] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:41.900442 13075 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.769336 13448 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:41.830122 13365 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:01:41.880723  8746 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:41.883420  8824 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:41.884106  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.825481  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.934528  8893 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:41.935544  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.823254 13404 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:41.896108  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.937788  8891 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.937956  8898 raft_consensus.cc:2376] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13. Re-sending same reply.
W20260709 13:01:41.895509  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 87190.9us
I20260709 13:01:41.882308  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.905076  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.937353  8749 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:41.817855 13376 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:01:41.817556 13395 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:01:41.823096 13361 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:01:41.905200 13426 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.876610 13248 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.949856  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.830521 13377 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:41.955004  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:41.938781  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.863193 13445 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:41.938987  8746 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:41.879873 13306 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.916925  8823 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:41.940423  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.942202  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.962378  8747 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:41.895438 13397 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.944267  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:41.939217  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:41.907596  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.945371  8897 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:41.956530  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.963852  8818 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:41.943742 13217 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:41.970402  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.970644  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.971050  8898 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:41.971163  8893 raft_consensus.cc:2376] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10. Re-sending same reply.
I20260709 13:01:41.971218  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.876932 13389 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:41.974793  8897 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:41.974664  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:41.957780  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.968765  8824 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:41.969508  8741 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
W20260709 13:01:41.955940 13415 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.001s
I20260709 13:01:41.976071  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.990653  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.991137  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.991460  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:41.986605  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:41.992224  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.991575  8893 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:41.993027  8749 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:41.992269  9447 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:41.993466  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:41.995069  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:41.995833  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:41.997422  8898 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:42.001482  9449 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:41.968318 13306 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.008478  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.010908  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.011694  8897 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:42.004194  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.013707  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:42.014633  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.015117  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.015967  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.016573  9447 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:42.018108  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.018117  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.018745  8749 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:41.978912 13389 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:41.998746 13422 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.002s
I20260709 13:01:42.020277  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.019620  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:42.026233 13453 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:01:42.026160  8818 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.033011  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:42.045671  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.036285  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:41.975534 13397 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.058094  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.007586 13448 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.061666  8747 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.056771  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.058791  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.059175  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:41.960984 13404 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.064955  8746 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:42.065003  8749 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:41.991813 13415 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.069248  8815 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:42.056771  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.070642 13429 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:42.071152  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.164718  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:42.164959  8746 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:42.036131 13422 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.066998 13448 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.078922  8747 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 10.
I20260709 13:01:42.035529 13305 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.013927 13393 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:42.136811 13454 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:01:42.126523  9474 heartbeater.cc:646] Failed to heartbeat to 127.7.117.190:45111 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.117.190:45111 timed out after 15.000s (SENT)
I20260709 13:01:42.021358 13402 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:42.145442  8820 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:42.057662 13361 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.140235  9442 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:01:42.124749 13453 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:42.161602  8818 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:42.086606  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.162936  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:42.207577  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 126327us
I20260709 13:01:42.207966  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.209270  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.071967 13392 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.065907  8898 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:42.167663 13430 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:42.167865 13460 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:01:42.169345 13409 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:42.150018 13449 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:42.174254  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:42.221585  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.175236  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:42.119076 13248 consensus_queue.cc:1175] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7fd110c312c14e17a2bbe6d453493cf2" responder_term: 10 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.6-9.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:42.178934  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.199766 13402 consensus_queue.cc:1243] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:01:42.165735 13465 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:01:42.175530 13456 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:01:42.175554 13448 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.190928 13422 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.160952 13397 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.193413 13426 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:42.194729 13394 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.849s	user 0.011s	sys 0.000s
I20260709 13:01:42.195500  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:42.171903 13459 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.197s	user 0.002s	sys 0.000s
I20260709 13:01:42.241413 13438 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.260160  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.260905  8889 raft_consensus.cc:2376] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14. Re-sending same reply.
I20260709 13:01:42.205227  8743 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:42.205585  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.160701 13075 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.264461 13075 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.216673  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.217433  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.265314  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:42.266296  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.267177  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:42.268350  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.259845 13438 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.179257  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.271044  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.266322  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.271809  8746 raft_consensus.cc:2368] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:42.272058  8897 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:42.272370  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.237609  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:42.238117  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59257.1us
I20260709 13:01:42.273209  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.209510 13398 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.226498 13457 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:42.277027  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.280467  8898 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:42.212414 13361 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.274765  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.282119  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.282974  8897 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:42.284255  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.269027  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:42.289140  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.290356  8746 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:42.291535  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.291198  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.293888  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.294680  8745 raft_consensus.cc:2376] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13. Re-sending same reply.
I20260709 13:01:42.295768  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.303357 13379 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.305820  8742 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:42.307353 13470 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:01:42.310464  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.311589  8889 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:01:42.314893 13409 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.321811 13423 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:42.322582 13423 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.326506 13416 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.327193 13416 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:42.330951 13433 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.636s	user 0.004s	sys 0.001s
W20260709 13:01:42.333443  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.334309  8744 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:42.288918 13419 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:42.335199 13419 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.336184 13361 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.337432 13457 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:42.337807  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.338037  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:42.338295 13457 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.287622 13386 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:01:42.343236 13426 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.343824  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.297024  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.344753  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.345420  8749 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:42.346011  8897 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:42.347726  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.351503  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.352332  8746 raft_consensus.cc:2376] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12. Re-sending same reply.
I20260709 13:01:42.338871  8898 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:42.357017  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.357919  8821 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:42.358865  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2395) took 1741 ms. Trace:
I20260709 13:01:42.359001  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.336380 13419 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.359004  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.359325  8892 rpcz_store.cc:276] 0709 13:01:40.617761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.686865 (+ 69104us) service_pool.cc:224] Handling call
0709 13:01:40.730204 (+ 43339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.973529 (+243325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.973580 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.974232 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.102376 (+128144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:41.102417 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.102480 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.330159 (+1227679us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:42.349943 (+ 19784us) raft_consensus.cc:1685] finished
0709 13:01:42.349974 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.358800 (+  8826us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92719,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":24523264,"thread_start_us":303669,"threads_started":2,"wal-append.queue_time_us":690837}
I20260709 13:01:42.360127  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.371930  8822 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:42.372543  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.304673 13306 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.374359  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.375149  8745 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.377691  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.377928 13379 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.378509  8749 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:42.323706 13423 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.379441  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.380734  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.382728  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.279070 13438 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.391443  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.392267  8817 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:42.345702 13416 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.407552  8817 pending_rounds.cc:85] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 7
I20260709 13:01:42.407871  8817 pending_rounds.cc:107] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:01:42.408089  8817 raft_consensus.cc:2889] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.397210  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.405133  8818 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:42.408728  8749 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:42.418377  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.424108  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.429999  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.430282  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.431015  8749 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.434563  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.436471  8822 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:42.436633  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.444725  8749 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:42.437484  8746 raft_consensus.cc:2376] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11. Re-sending same reply.
I20260709 13:01:42.465112  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.466709  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.467442  8821 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.467552  9443 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:42.466643  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 182523us
I20260709 13:01:42.466842  8749 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:42.470476  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.470714  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.471243  8823 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:42.468783  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.475459 13372 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.475962 13372 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.476364 13417 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:42.477993 13372 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.479640  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.479879  8821 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:42.480278  8819 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:42.480824  8745 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:42.484716  8746 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:01:42.487305  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.487911  8746 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:42.490592  8819 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:42.494102  8749 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:42.496706  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.497808  8822 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:42.496343  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.483496 13406 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.493016  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.510370  8746 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.512764  8819 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:42.501577  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.503798  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:42.504237  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.504920  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.514034  8821 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.507330  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.515374  8823 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:42.521257  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.529940  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:42.530392  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.493618 13361 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.530841  8746 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:42.493417 13386 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.482844 13389 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.532233  8749 raft_consensus.cc:2376] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12. Re-sending same reply.
I20260709 13:01:42.509873 13406 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.548069  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:42.550944  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46790.6us
I20260709 13:01:42.532059  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.532919 13397 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.561038  8821 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:42.564019  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.564844  8749 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:42.565943  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.552621  8746 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:42.535830 13429 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:01:42.563844  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:42.553959  8863 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:42.554548 13476 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:42.558346  8822 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:42.534402 13407 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:42.553704 13472 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:01:42.572286  8749 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:42.573987  8895 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:42.576479  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.561431 13428 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.534574 13430 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:42.590037  8746 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12. Re-sending same reply.
I20260709 13:01:42.562538 13406 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.573666 13397 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.533191 13372 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:01:42.535612 13361 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.541431 13386 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.592104  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.575294 13476 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.582209 13305 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.594710  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.654412  8746 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.657038  8749 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:42.657883 13399 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:42.659458 13415 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.692021 13399 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.694592 13409 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.694602 13399 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.695221 13409 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.695636 13399 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.698912 13361 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.580257 13435 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:42.661818 13477 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:01:42.664316 13414 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:42.700685 13414 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.667586  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.703613  8892 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.705555 13435 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.705646  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.668287  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.707252  8898 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.708400  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.711490  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.712814  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:42.713214  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.714066  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.714329  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.715206  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.668493 13398 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.668946  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:42.670445 13386 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.601477 13407 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.649458 13397 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.680230 13413 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:42.684702  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 755) took 1251 ms. Trace:
I20260709 13:01:42.729242  8892 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:42.737996  8890 rpcz_store.cc:276] 0709 13:01:41.433214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.433795 (+   581us) service_pool.cc:224] Handling call
0709 13:01:41.469007 (+ 35212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.677251 (+208244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.684376 (+1007125us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:42.684626 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389159296}
W20260709 13:01:42.685789 13473 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:01:42.676888  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.721750 13361 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:42.653546 13428 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:42.686683  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.648464 13306 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.716060 13409 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.706655  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.716789  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.736404  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.743297  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.732091  8898 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:42.743799  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:42.744355  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.736742  8746 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:42.714916  8749 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:42.739751 13448 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.745911  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:42.722946  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:42.747298 13448 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.722033  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.747710  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.742300  8748 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12. Re-sending same reply.
I20260709 13:01:42.749187  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.749238  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:42.685810  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.741783  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 178082us
I20260709 13:01:42.736670  9448 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:42.722313 13372 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.712393 13417 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.752796 13417 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:42.691707  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.754714  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.755446  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:42.756464  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.728531 13390 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:42.688954 13476 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.747517 13429 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:42.748049  9446 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.740553  8741 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:01:42.764364  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.765425  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.749831  8748 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:42.753093 13372 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.766531 13372 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.766633  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:42.766955 13372 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.767331 13372 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.767678  8748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "011b370821004185a003be210a525c01"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:01:42.759734  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:42.762017  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.769181 13416 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:01:42.771421 13424 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:42.773594  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.774433  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:42.737866 13413 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.775194  9449 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:42.775261  8746 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:42.775993  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 872) took 1706 ms. Trace:
I20260709 13:01:42.761924 13394 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:42.733338 13491 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.758733 13395 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.706635 13435 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.763943 13268 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:42.787241  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.777371 13432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.018s	sys 0.019s
W20260709 13:01:42.806488  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44586.7us
I20260709 13:01:42.769019 13386 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.769268 13422 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:42.796633  8748 raft_consensus.cc:606] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.769495 13429 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:42.779956  9440 rpcz_store.cc:276] 0709 13:01:41.070131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.070272 (+   141us) service_pool.cc:224] Handling call
0709 13:01:41.070618 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.070868 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:41.354357 (+283489us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.631020 (+276663us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.641420 (+ 10400us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.643594 (+  2174us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.185659 (+542065us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b180048f040. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:42.590552 (+404893us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b180048f040. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:01:42.775624 (+185072us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.775674 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:42.775720 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.775831 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.775941 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8183,"prepare.queue_time_us":543551,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":5573,"raft.queue_time_us":24470,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":18004,"spinlock_wait_cycles":63102720,"thread_start_us":703273,"threads_started":3,"wal-append.queue_time_us":451645}
I20260709 13:01:42.809770  8748 raft_consensus.cc:614] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:42.762272 13488 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:42.808116  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.808672  8862 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:42.811189  8862 leader_election.cc:403] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:42.769032 13375 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:01:42.811805 13475 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.773661 13390 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.815971  9445 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:42.771800 13424 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.781915 12889 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.775075 13389 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.813711  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:42.783113 13394 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:42.806887 13432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.019s	sys 0.019s
I20260709 13:01:42.793133 13368 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.815766 13491 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.796195 13430 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.815526 13429 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:42.832624  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 76240us
I20260709 13:01:42.835306  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:42.823741 13395 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.819540 13428 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.825105 13422 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:42.827124 13394 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:42.853168  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44688.7us
I20260709 13:01:42.827343 13432 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:42.828387 13361 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:42.828260 13268 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:42.854463 13375 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.810114 13407 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.865079 13375 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.830302 13413 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.883806  9446 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:42.880533  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:42.886852  8891 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:01:42.888695  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:42.867671 13485 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:01:42.852494 13395 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.858089 13424 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.905817 13424 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.908207 13394 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.874606 13422 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.910354  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.911113  8820 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.911271 13379 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:42.911751 13379 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.852731 13438 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:42.894613 13447 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.199s	user 0.000s	sys 0.004s
I20260709 13:01:42.894637 13406 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:42.896449 13483 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.239s	user 0.000s	sys 0.002s
I20260709 13:01:42.903045 13417 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.904809 13261 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.866696 13498 raft_consensus.cc:2764] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:42.872059 13375 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.913437  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.929782  8815 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:42.923378  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:42.924935  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89740.4us
I20260709 13:01:42.925454  8891 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:42.930505  8822 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.930584  8891 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.930795  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.926064  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:42.910403 13452 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.213s	user 0.000s	sys 0.003s
I20260709 13:01:42.932246  8818 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:42.923197 13379 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.905058 13476 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.928822  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.892237 13419 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:42.931452  8891 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.927827  8819 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:01:42.945801  8819 pending_rounds.cc:85] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 8
I20260709 13:01:42.946072  8819 pending_rounds.cc:107] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:01:42.946291  8819 raft_consensus.cc:2889] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.945401  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:42.947135  8815 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:01:42.936794  8820 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:42.938305  8817 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.938423  8816 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:42.933085 13379 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.954826 13476 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.956429 13395 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:42.878108 13361 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.958958  8817 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:42.941658 13406 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:42.932893 13435 consensus_queue.cc:1175] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "112105225cc140459688be5d31dcb071" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 12 index: 10 } }
I20260709 13:01:42.943534 13248 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:42.965965  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:42.945359  8891 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.863550 13390 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.952507 13394 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.913326 13428 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.875715 13268 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.972210 13502 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:42.973053 13502 consensus_queue.cc:1243] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:42.972290 13368 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.973558 13497 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:42.973647  8893 raft_consensus.cc:2376] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10. Re-sending same reply.
I20260709 13:01:42.974210 13497 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:42.974380 13248 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:42.929260 13374 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:42.967000  8820 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.969053  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2187) took 1153 ms. Trace:
I20260709 13:01:42.933511 13417 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.971377 13490 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.971655 13419 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:42.986168 13248 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.991138 13438 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:42.991055 13419 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.972110 13498 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:42.973793 13368 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:42.966346  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 875) took 1717 ms. Trace:
I20260709 13:01:42.996446  9444 rpcz_store.cc:276] 0709 13:01:41.248571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.249372 (+   801us) service_pool.cc:224] Handling call
0709 13:01:41.249727 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.250155 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.250193 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.250682 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.476249 (+225567us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:41.476312 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.476410 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.891739 (+1415329us) spinlock_profiling.cc:243] Waited 9.95 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:42.960103 (+ 68364us) raft_consensus.cc:1685] finished
0709 13:01:42.960133 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.966283 (+  6150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2274,"raft.queue_time_us":1673,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":14992,"spinlock_wait_cycles":22886144,"thread_start_us":225197,"threads_started":1,"wal-append.queue_time_us":423855}
I20260709 13:01:42.976655  8789 leader_election.cc:400] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:42.977344  8891 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:42.983371  8822 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:42.985199  8889 rpcz_store.cc:276] 0709 13:01:41.815483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.312307 (+496824us) service_pool.cc:224] Handling call
0709 13:01:42.312742 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.448711 (+135969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.449501 (+   790us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.471919 (+ 22418us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.572308 (+100389us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:42.572350 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.572420 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.968853 (+396433us) raft_consensus.cc:1685] finished
0709 13:01:42.968872 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.968996 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29234,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":213664,"threads_started":2,"wal-append.queue_time_us":278686}
I20260709 13:01:42.888489 13432 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:42.987849 13502 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:01:42.999307  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.000082  8891 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.998359  8789 leader_election.cc:403] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.001096  8820 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:43.001986  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.007025  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.008119  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.008129  8891 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:43.005192 13394 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:42.970192 13395 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:01:43.021248  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 921) took 1026 ms. Trace:
I20260709 13:01:43.021914  9442 rpcz_store.cc:276] 0709 13:01:41.994727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.994845 (+   118us) service_pool.cc:224] Handling call
0709 13:01:43.020976 (+1026131us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54005f0fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.021162 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332112768}
I20260709 13:01:43.026126  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:42.994973 13497 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.011740 13499 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.012130 13422 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.120090  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2407) took 1178 ms. Trace:
I20260709 13:01:43.030555 13394 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.049566 13438 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:43.111923 13462 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.091s	user 0.000s	sys 0.006s
I20260709 13:01:43.035537 12889 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:43.132967 12889 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.030777 13414 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.135491 13429 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.129833 13394 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.137912 13394 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.115149  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2186) took 1299 ms. Trace:
I20260709 13:01:43.120683  8893 rpcz_store.cc:276] 0709 13:01:41.941448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.999602 (+ 58154us) service_pool.cc:224] Handling call
0709 13:01:43.045880 (+1046278us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540015d954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.120012 (+ 74132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175483136}
I20260709 13:01:43.110350  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 752) took 1740 ms. Trace:
I20260709 13:01:43.123378  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.144475  8894 rpcz_store.cc:276] 0709 13:01:41.369915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.370028 (+   113us) service_pool.cc:224] Handling call
0709 13:01:41.433752 (+ 63724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.828586 (+394834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.040867 (+1212281us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540015f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:43.110273 (+ 69406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861171072}
I20260709 13:01:43.131405 13424 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.131759 13393 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:43.130357  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.152276  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.145151  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.145951  8895 rpcz_store.cc:276] 0709 13:01:41.815223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.815325 (+   102us) service_pool.cc:224] Handling call
0709 13:01:43.045878 (+1230553us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540015de54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.115039 (+ 69161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562058112}
I20260709 13:01:43.155017  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:43.161389 13466 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.988s	user 0.004s	sys 0.003s
I20260709 13:01:43.164446 13429 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.164991 13217 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:43.158660 13426 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:43.173110  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.158159 13435 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:43.162907 13497 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.161499 13438 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.149219 13424 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:43.184587 13426 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.158264 13372 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.195179 13413 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:43.209178 13492 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.451s	user 0.004s	sys 0.000s
I20260709 13:01:43.211793 13426 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.212858 13397 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.220541 13475 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:43.251789  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.253654  8745 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:42.944360  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:43.255857  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.179796 13268 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:01:43.174005 13217 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.259032  8749 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:43.286501 13491 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.265367 13248 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.262198  8746 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:43.288084 13248 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.304081 13248 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.272518 13217 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:43.258863  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.270185 13268 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:43.296808 13429 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.310750 13268 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.311055 13429 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:43.261775  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101011us
I20260709 13:01:43.274708 13372 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:43.312872 13268 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.287779 13491 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:43.270030  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96872.5us
I20260709 13:01:43.317217 13491 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.321359  9441 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:43.327389 13457 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.328258 13457 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.270879 13368 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.330389  8745 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:43.302233 13424 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.275278 13305 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:43.331835 13305 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:43.257206  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 312813us
I20260709 13:01:43.335858  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:43.311375 12889 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.343371 13429 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.344373 12889 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:43.354249  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 346107us
W20260709 13:01:43.354624  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.361377  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.355162  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 344110us
I20260709 13:01:43.363801 13379 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:43.364969 13379 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.325120 13261 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:01:43.368554 13379 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.374123  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.383587  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.389986  8747 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:43.386158  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2125) took 1455 ms. Trace:
I20260709 13:01:43.396227  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 922) took 1400 ms. Trace:
I20260709 13:01:43.396724  9443 rpcz_store.cc:276] 0709 13:01:41.994994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.995105 (+   111us) service_pool.cc:224] Handling call
0709 13:01:41.995324 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.468029 (+472705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.392947 (+924918us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54005f23d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:43.395384 (+  2437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085873920}
I20260709 13:01:43.398414 13497 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:43.399925  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.402863 13488 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:43.403139  8818 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:43.399344 13497 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.403980  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 819) took 1230 ms. Trace:
I20260709 13:01:43.405514  9447 rpcz_store.cc:276] 0709 13:01:42.173695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.173876 (+   181us) service_pool.cc:224] Handling call
0709 13:01:43.403481 (+1229605us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54001937d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.403744 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2039814912}
I20260709 13:01:43.406767  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:43.407397  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.411204  8821 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:43.412021 13432 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:43.412511 13432 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.408854  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:43.413935  8816 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.413908 13432 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.415416 13432 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.415629  8893 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:01:43.415968  8893 pending_rounds.cc:85] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:01:43.416203  8893 pending_rounds.cc:107] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:01:43.416479  8893 raft_consensus.cc:2889] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.418560 13500 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:43.423465  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.424754  8820 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:43.425855  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:43.418375  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.428586  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.434554  8747 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:43.435730  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.433063 13520 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.436594  8747 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.437525 13261 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.437913 13248 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.438441 13261 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.439642 13404 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.440426 13248 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:43.442332 13261 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:43.443349 13508 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
I20260709 13:01:43.445916  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.446848  8749 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.430056 13497 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.433256  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.452642 13497 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.455305  8890 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:43.418229  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44332.1us
I20260709 13:01:43.464682  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.466599 13414 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.391392  8742 rpcz_store.cc:276] 0709 13:01:41.930552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.930940 (+   388us) service_pool.cc:224] Handling call
0709 13:01:43.385552 (+1454612us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54000b5c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.385906 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859608320}
I20260709 13:01:43.467598 13414 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.468422  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.469359  8742 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:43.470683  8742 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:01:43.473768  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.473937  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:43.475427 13368 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.475924 13368 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.476255 13368 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.403719 13488 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:43.478626  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2118) took 1672 ms (client timeout 1547 ms). Trace:
W20260709 13:01:43.479167  8744 rpcz_store.cc:269] 0709 13:01:41.806443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.806619 (+   176us) service_pool.cc:224] Handling call
0709 13:01:43.478230 (+1671611us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54001c8254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.478447 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151710464}
I20260709 13:01:43.478677  8822 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:43.480281  8822 pending_rounds.cc:85] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 5
I20260709 13:01:43.480587  8822 pending_rounds.cc:107] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:01:43.480875  8822 raft_consensus.cc:2889] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.480970  8744 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:43.488574  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1852) took 1004 ms. Trace:
I20260709 13:01:43.489059  8744 rpcz_store.cc:276] 0709 13:01:42.484476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.480169 (+995693us) service_pool.cc:224] Handling call
0709 13:01:43.480621 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.488254 (+  7633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.488478 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.491199  8744 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:43.473510  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:43.487690  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2122) took 1655 ms (client timeout 1556 ms). Trace:
I20260709 13:01:43.493563 13399 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.494515 13417 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:43.495496 13438 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:43.498289  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.492871 13475 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.498826  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 779) took 1164 ms. Trace:
I20260709 13:01:43.499402  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.512317  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2163) took 1166 ms. Trace:
I20260709 13:01:43.515127  8742 rpcz_store.cc:276] 0709 13:01:42.341771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.470019 (+1128248us) service_pool.cc:224] Handling call
0709 13:01:43.470366 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.470971 (+   605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.507700 (+ 36729us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.516812 13417 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.520473 13417 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.510269  8746 rpcz_store.cc:276] 0709 13:01:42.334436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.785526 (+451090us) service_pool.cc:224] Handling call
0709 13:01:42.785835 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.295885 (+510050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.498441 (+202556us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400446bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:43.498728 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094007680}
I20260709 13:01:43.521804  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1856) took 1024 ms. Trace:
I20260709 13:01:43.522239  8744 rpcz_store.cc:276] 0709 13:01:42.488407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.490276 (+1001869us) service_pool.cc:224] Handling call
0709 13:01:43.490761 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.509648 (+ 18887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.513139 (+  3491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.524852  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:43.525882  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61572.8us
I20260709 13:01:43.527614 13419 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:43.528286  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.529603  8823 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
W20260709 13:01:43.500097  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67048.3us
I20260709 13:01:43.501883  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.503278  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.485014 13414 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:43.504374  8743 rpcz_store.cc:269] 0709 13:01:41.831738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.831907 (+   169us) service_pool.cc:224] Handling call
0709 13:01:43.487301 (+1655394us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54001d9054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:43.487514 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531529088}
I20260709 13:01:43.534925  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:43.529551  8897 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:43.535712  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.546003  8744 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:43.546228  8816 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:43.550379  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:43.533200 13488 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.501101 13394 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:43.560734  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.560259  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:43.541182 13399 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.554733 13438 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.573123  8822 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.576354  8744 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:43.576257  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:43.577847  8744 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.559693  8895 raft_consensus.cc:2376] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10. Re-sending same reply.
I20260709 13:01:43.580226  8890 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:43.582736  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.585060  8745 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:43.562906 13414 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.594616 13414 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.597311 13361 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:43.598156 13361 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.601612  8821 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.602625 13248 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:43.603883 13457 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.610819  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.611056  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.611868  8816 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:43.603969 13401 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.622999  8749 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:43.625641  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.617105 13497 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.638007  8744 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:43.640398  8747 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:43.623454 13520 raft_consensus.cc:2764] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:01:43.572535 13475 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.650882 13423 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.651394 13423 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.651823 13423 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.653854 13394 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:43.656059  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 107272us
I20260709 13:01:43.661734 13415 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:43.662120  8890 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:43.662282 13415 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.663318  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.663285 13415 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.664062  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.664798  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.665501  8748 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 12. Ops: [10.6-10.6]
I20260709 13:01:43.670635  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.679867  8898 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:43.680940  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.671140 13475 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.671602  8896 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.684356 13399 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.685429 13415 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.673072 13379 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.685868 13438 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.687505 13379 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:43.690026  8743 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:43.694133  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 780) took 1009 ms. Trace:
I20260709 13:01:43.694248  8894 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:43.694662  8746 rpcz_store.cc:276] 0709 13:01:42.684653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.525093 (+840440us) service_pool.cc:224] Handling call
0709 13:01:43.553482 (+ 28389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.595798 (+ 42316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.595827 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.595886 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:43.595916 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.693891 (+ 97975us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b5400359154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:43.693936 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.694046 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96753024}
I20260709 13:01:43.695516  8864 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13.
I20260709 13:01:43.696915  8817 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:01:43.697885  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.700079 13523 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.133s	user 0.000s	sys 0.003s
I20260709 13:01:43.701110  9442 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:43.701112  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:43.701737  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.701898  9447 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:43.701996  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:43.702627  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.703114  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.703926  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.705260 13424 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:43.706199  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.707268 13305 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.709694  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:43.722880  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.709797  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.710323  8823 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:43.639313 13488 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:43.649461  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:43.677650  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 117601us
I20260709 13:01:43.715540  9448 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:43.718878  8896 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:43.649778 13497 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.726292  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.727897  8821 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:43.728430  8822 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:43.715274  9444 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:43.733412  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.734169  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:43.731020  9449 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:43.732630  9440 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:43.719754  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.744401  8889 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
W20260709 13:01:43.747331  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50234.2us
I20260709 13:01:43.750882  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.762306  8890 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:43.762926  9442 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:43.761655  8820 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:43.639815 13401 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.710369 13393 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:43.792181  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:43.774199 13520 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.776485 13497 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:43.763198 13404 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:01:43.786568 13397 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:43.788756 13305 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.792627 13368 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:43.736214 13438 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:01:43.770169  8891 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:43.805763  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.810427  8740 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:43.810925  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.811991  8894 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.814801 13390 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:43.808581 13432 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:43.817708 13432 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.818084 13432 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.819212 13407 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.819687  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:43.821076 13428 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:43.824313 13413 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:43.801052 13268 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:01:43.825557  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 179573us
I20260709 13:01:43.823496  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.830349 13361 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:43.830685  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.831022 13268 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:43.831410 13268 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:43.832413 13268 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.833572 13361 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.836908  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:43.837845  8889 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:01:43.839442 13401 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.817178 13404 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:43.819718 13407 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.850605 13404 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:43.850893  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 90680.8us
I20260709 13:01:43.842489  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:43.843149  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1863) took 1351 ms. Trace:
W20260709 13:01:43.852420  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84702.5us
I20260709 13:01:43.844501  9442 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:43.845512  8743 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:43.844892  8894 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:43.833014  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.849028  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:43.856278  8744 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:43.771638 13488 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.851779  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:43.861338  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.864959  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.858724  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.864809  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.868402  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:43.873340  8898 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:43.848937 13390 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.874627  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.876497  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.860040 13497 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:43.879478  9448 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:43.880276  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.880482  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.881512  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.870819 13520 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.882264 13498 raft_consensus.cc:2764] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:01:43.882460  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.882100 13404 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.882824  9445 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:43.883333 13438 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:43.885241  8744 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:43.867067 13248 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:43.886117  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53229.1us
I20260709 13:01:43.886469  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.885383  8749 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:43.895718  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.898767  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.899443  8743 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:43.901044  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:43.887291 13497 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.890767 13520 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.887158 13361 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:43.900473  8745 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:43.911155  8747 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:01:43.852344  8742 rpcz_store.cc:276] 0709 13:01:42.491812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.531085 (+1039273us) service_pool.cc:224] Handling call
0709 13:01:43.565671 (+ 34586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.597778 (+ 32107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.597814 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.598423 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.638515 (+ 40092us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:43.641418 (+  2903us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.641872 (+   454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.641944 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.842921 (+200977us) raft_consensus.cc:1685] finished
0709 13:01:43.842939 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.843075 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5085,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":5919,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":2165,"thread_start_us":39645,"threads_started":1,"wal-append.queue_time_us":147523}
I20260709 13:01:43.909206  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:43.901199 13268 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.917132  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:43.902716 13248 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:43.910905 13497 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.922804 13435 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:43.923513 13511 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.913353 13361 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.923957 13521 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.936750  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:43.937093 13521 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:43.900754 13217 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:01:43.926889  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:43.927011  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:43.929484 13424 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:43.901302 13419 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:43.931481  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:43.923125 13248 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.913046 13520 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.925406 13488 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.918123 13394 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.914944 13438 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:43.940271  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.923957 13399 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:43.946775  8891 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:43.958709  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:43.948114 13404 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.961733  8893 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 13 because replica has last-logged OpId of term: 11 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 12.
I20260709 13:01:43.950037  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.938571 13361 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:43.935585 13475 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:43.938201 13500 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.957477 13457 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:43.967171 13413 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:43.967615 13535 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:43.968076 13413 consensus_queue.cc:1243] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:43.968294 13535 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:43.968995 13437 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:43.969435 13437 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:43.970841 13535 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:43.970680 13437 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:43.957644  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:43.972529  8896 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.952493 13394 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.977576 13533 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:01:43.978142 13394 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.946137 13424 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.958534 13399 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:43.987320 13399 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.950569 13305 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:01:43.942258 13521 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:43.982816  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.986022  8745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "318340b5bf3b4640972d271299414f94"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:01:43.986465  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:43.986755  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.999773  8745 raft_consensus.cc:606] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:01:44.015959  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:44.016279  8896 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:44.030323  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78592.5us
I20260709 13:01:44.034247  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.027612  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:43.960879 13488 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:01:44.071530  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1851) took 1587 ms. Trace:
I20260709 13:01:44.077452  8892 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:44.073902  8742 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:44.072142  8748 rpcz_store.cc:276] 0709 13:01:42.484261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.471619 (+987358us) service_pool.cc:224] Handling call
0709 13:01:43.641610 (+169991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.679241 (+ 37631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.063956 (+384715us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400021cd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:44.071441 (+  7485us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388695296}
W20260709 13:01:44.076504  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 149704us
I20260709 13:01:44.028932 13397 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.081820  8749 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:44.031668 13305 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.031118 13368 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.032259 13475 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:44.089116  8744 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:44.076711  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2123) took 2226 ms (client timeout 2192 ms). Trace:
I20260709 13:01:44.081703  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.063586 13498 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:44.092729  8740 rpcz_store.cc:269] 0709 13:01:41.836837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.904888 (+ 68051us) service_pool.cc:224] Handling call
0709 13:01:44.062425 (+2157537us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540013b0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:44.062624 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4009290112}
I20260709 13:01:44.089881 13457 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:44.096452  8824 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:01:44.095921  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 148013us
I20260709 13:01:44.096051  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1847) took 1602 ms. Trace:
I20260709 13:01:44.080653 13435 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.113705  8891 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:44.082516 13511 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.123134  8741 rpcz_store.cc:276] 0709 13:01:42.483136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.765128 (+281992us) service_pool.cc:224] Handling call
0709 13:01:44.039187 (+1274059us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54001d8b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:44.085263 (+ 46076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217569152}
I20260709 13:01:44.120900  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:44.129427  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.120733 13217 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:01:44.131069  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:44.133208  8741 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:01:44.133885 13536 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:01:44.135138 13543 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.137156 13435 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.106500 13406 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.199s
I20260709 13:01:44.104674 13305 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.143649  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:44.144299  8741 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.146292  8891 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:01:44.148732 13463 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.133s
W20260709 13:01:44.150641 13534 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:01:44.152112  8741 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:44.152951  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:44.153364  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:44.154929  8893 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:01:44.158248  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49402.4us
I20260709 13:01:44.159250  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:44.130625  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 170787us
I20260709 13:01:44.164582  8746 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:44.122745 13498 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:44.168759 13498 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.169921 13432 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.090370 13397 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.160490 13488 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:44.199559  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 133219us
I20260709 13:01:44.200345  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.177480 13521 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:44.109942 13438 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.207562  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.166536 13217 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:01:44.214174  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:44.173702 13406 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:01:44.215566  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.217005  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.220947  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.221158  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:44.223453  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64340.5us
I20260709 13:01:44.229384  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.230352  8821 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:44.231662  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 304767us
I20260709 13:01:44.202555 13488 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:44.242344  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.187023 13463 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:01:44.243654  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45904.5us
I20260709 13:01:44.211611 13248 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.218281 13521 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:44.222499 13414 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:44.261170  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47620.9us
I20260709 13:01:44.247254 13404 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.247001 13500 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.250946 13390 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.279659  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:44.246619 13547 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 13:01:44.291133  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 84605.8us
I20260709 13:01:44.307083 13217 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:44.307801  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.308560  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64193.6us
I20260709 13:01:44.309257  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:44.283365 13361 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.277503 13406 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:01:44.314272 13217 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:44.314827 13217 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.315230 13217 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:44.242249 13475 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.268505 13248 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:01:44.302505 13432 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.287380 13401 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:44.250038 13457 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:44.350687  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 83477.7us
W20260709 13:01:44.363803 13548 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:01:44.365139 13541 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:01:44.375828  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 126202us
I20260709 13:01:44.376977  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:44.383883  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.355108 13500 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.335611 13399 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.396327 13397 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.397809 13558 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:44.398227 13558 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:44.399348 13558 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.392051 13414 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.323472 13488 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.403038 13488 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.406136  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:44.385782 13361 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:44.413601  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.414803  8893 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:44.408879 13537 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 13:01:44.416256  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47549.2us
I20260709 13:01:44.420110  8895 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:01:44.419101 13423 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.767s	user 0.004s	sys 0.010s
W20260709 13:01:44.426064  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 193694us
I20260709 13:01:44.412145 13558 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.435132 13545 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:01:44.439440  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:44.433046 13500 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.439131 13551 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:44.455377 13551 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.456333 13500 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.457623 13401 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.437045 13549 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.139s
I20260709 13:01:44.461380 13544 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:44.462098 13556 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:44.462350 13404 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:44.463909  8815 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:44.463434 13397 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:01:44.392658 13521 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.438699 13423 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.493700 13368 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:01:44.516873  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 82165us
W20260709 13:01:44.512835 13552 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:01:44.526973 13423 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:44.532792 13394 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.004s
I20260709 13:01:44.534338 13399 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.496183 13361 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:44.545576  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 109654us
I20260709 13:01:44.499003 13401 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.519497 13397 consensus_queue.cc:1243] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:01:44.554967  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59054us
W20260709 13:01:44.546830 13497 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.006s
I20260709 13:01:44.513965 13555 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:01:44.563259 13497 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.563745 13401 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:44.566056 13568 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:44.566851 13497 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:44.573426 13424 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.003s
I20260709 13:01:44.573988 13424 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.576462 13424 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.499457 13546 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.196s
I20260709 13:01:44.579771 13217 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.561220  8787 leader_election.cc:400] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:01:44.518811 13551 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.582332  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2202) took 1001 ms. Trace:
I20260709 13:01:44.586616  8897 rpcz_store.cc:276] 0709 13:01:43.580990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.581157 (+   167us) service_pool.cc:224] Handling call
0709 13:01:43.642039 (+ 60882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.873048 (+231009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.873079 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.880567 (+  7488us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.906361 (+ 25794us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:43.906396 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.906460 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.581997 (+675537us) raft_consensus.cc:1685] finished
0709 13:01:44.582027 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.582223 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1785,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"thread_start_us":211362,"threads_started":2,"wal-append.queue_time_us":65670}
I20260709 13:01:44.551492 13361 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:44.591884  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 149357us
I20260709 13:01:44.593219 13361 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.602162  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.596122  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:44.606731  8896 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:44.595163  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45063.2us
I20260709 13:01:44.552029 13394 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.625857  9445 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:44.598217 13520 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.003s
I20260709 13:01:44.521257 13556 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.654208  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.628901 13545 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:44.659551  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:44.596900 13544 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:44.675482 13556 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.694383 13217 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.699818 13576 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.700714  8894 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:44.700860  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:44.702205  8896 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:44.707027 13576 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.710428  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:44.711352  8891 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:44.733803  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 137825us
I20260709 13:01:44.634718 13520 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:44.770737  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.772310 13520 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.774439 13394 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.776912  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.778757  8898 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:44.779498  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:44.780551  8748 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.763042 13521 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.764281 13544 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.789479  8898 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:44.789942 13561 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.792028 13248 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.800567 13437 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:44.797871 13248 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.806363 13248 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.795415 13544 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.798429 13561 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.811278  8787 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:44.808853 13544 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.819823 13520 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.826890  8891 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:44.793699 13521 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.817485 13544 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:44.860147 13570 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.318s	user 0.003s	sys 0.011s
I20260709 13:01:44.825800 13497 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:01:44.870857 13435 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:44.870540 13544 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.879534 13521 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:44.880668 13569 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:01:44.881156 13561 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.882300 13497 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:44.882773 13497 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:44.883728 13497 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:44.888878  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.889612  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.890054  8816 raft_consensus.cc:2410] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:44.890146 13406 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:44.890789 13406 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.891121  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.873253 13305 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:44.859532 13576 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.893644  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:44.894106  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.889856  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.885066 13497 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.892861 13490 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.900421 13490 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.892953 13406 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:44.873207 13544 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.907543  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:44.907809  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:44.913051  8821 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:44.916487  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:44.919348  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:44.894423 13361 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.918691  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:44.919412  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:44.918882  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:44.923386  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:44.925700  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:44.925844  9447 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:44.923913 13305 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:44.927047 13305 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.927414 13305 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:44.930449  9443 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:44.930450  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:44.931296  8898 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:44.931903 13585 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:01:44.923476  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:44.932724  9442 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:44.933662 13419 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.934175 13419 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.935843  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.936810  8891 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:44.938607  9445 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:44.925190  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:44.940721  8748 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.938042 13490 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.946796  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.946238  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:44.953814  8898 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:44.954396  8895 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.955235  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.955277  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:44.956460  9448 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.963976  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:44.936244 13419 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.965163  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:44.967985  9445 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:44.966383  9443 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.970323  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:44.971141  9442 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.977140  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.978811  8891 raft_consensus.cc:1217] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-14.8]   Dedup: 8.5->[10.6-14.8]
W20260709 13:01:44.981526  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.984484  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:44.953732 13428 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.985944 13428 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.988528 13428 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.967288 13438 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:44.989678  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:44.989898 13438 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:44.992328 13438 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:44.998224  8748 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:44.999380  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.000627  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.001601  8743 raft_consensus.cc:2410] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:44.979285  8891 pending_rounds.cc:85] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 5
I20260709 13:01:45.006346  8891 pending_rounds.cc:107] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:45.006794  8891 raft_consensus.cc:2889] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:45.002249 13393 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:45.008219  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:45.008867  8748 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:45.012853  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2211) took 1351 ms. Trace:
I20260709 13:01:45.015916  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.018298  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.019390  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.013507  8890 rpcz_store.cc:276] 0709 13:01:43.655743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.692661 (+ 36918us) service_pool.cc:224] Handling call
0709 13:01:43.693020 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.765067 (+ 72047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.006533 (+1241466us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b540045a6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:45.006723 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134519168}
I20260709 13:01:45.020153  8744 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:01:45.019533 13581 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.312s	user 0.004s	sys 0.000s
I20260709 13:01:45.023469  8748 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:45.024935  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:45.029779  8823 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:45.029439  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:45.063495  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:45.008177 13393 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:45.087657  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:45.090351  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:45.031872 13432 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:45.093717  8744 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.098342 13361 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.099157 13488 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.099789 13488 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:45.102797  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 118421us
I20260709 13:01:45.112471  8744 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:45.108793  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:45.116252  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86917us
I20260709 13:01:45.088502 13475 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.118211  8823 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:45.121027  9445 raft_consensus.cc:2368] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:45.122044  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:45.123266  8821 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:45.124805  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.127244  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.127439  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:45.128101  8819 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:45.129247  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:45.130934 13549 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:45.131563  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:45.132699 13261 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:45.133183 13261 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:45.106381 13372 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:45.136744  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.133025  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:45.108752 13428 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:45.134130 13261 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.139830  9443 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:45.140312 13589 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:01:45.118173 13379 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.142380 13457 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:45.142009  8889 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:45.152001  8821 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:45.154021  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.154238  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:45.155323  8823 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:45.116845 13551 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:01:45.157409  8819 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:45.164939  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:45.165421  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.166056  8741 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:45.166913  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.169174  8821 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:45.171140  8748 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:45.156525 13562 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:45.179234  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.180241  8741 raft_consensus.cc:2410] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:45.162655 13429 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.182286  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.182034  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:45.162094 13488 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.163424 13457 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:45.149708 13475 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.206766  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:45.145749 13372 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.183897  9441 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:45.121793 13393 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.157637 13379 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.296136  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.286166  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:45.297240  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 114057us
I20260709 13:01:45.273550  8788 leader_election.cc:400] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:45.157464 13428 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.275156  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:45.275672 13546 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.277724  8863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.603s (SENT) [suppressed 58 similar messages]
I20260709 13:01:45.296782  8823 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:45.262893  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:45.303834  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.297977  8788 leader_election.cc:403] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:45.199952  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:45.301393  8889 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:45.305712  8821 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:45.266315 13562 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:45.307055 13562 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.308374  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.310655  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.311403  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1876) took 1344 ms. Trace:
I20260709 13:01:45.311694  8747 rpcz_store.cc:276] 0709 13:01:43.966916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.310420 (+1343504us) service_pool.cc:224] Handling call
0709 13:01:45.311346 (+   926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.312659  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.313732  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 782) took 1477 ms. Trace:
I20260709 13:01:45.314105  8746 rpcz_store.cc:276] 0709 13:01:43.836303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.848102 (+ 11799us) service_pool.cc:224] Handling call
0709 13:01:45.313559 (+1465457us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:45.313680 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674495360}
I20260709 13:01:45.315071  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:45.206568 13429 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.315997  8747 raft_consensus.cc:2368] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:45.317075  9445 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:01:45.317476  9445 pending_rounds.cc:85] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 10
I20260709 13:01:45.317682  9445 pending_rounds.cc:107] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:01:45.317886  9445 raft_consensus.cc:2889] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:45.320137  8821 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
W20260709 13:01:45.324503  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61686.5us
W20260709 13:01:45.324714  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 59716.6us
I20260709 13:01:45.325155  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:45.326033  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:45.332190  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:45.338840 13379 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:45.338646  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60964.7us
I20260709 13:01:45.342473  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:45.301788 13556 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:45.302588 13597 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:45.351691  8746 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:45.330004 13561 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.299348 13305 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.331037 13576 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:45.344916 13544 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.304553 13572 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:01:45.339780 13551 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.373550  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.326082 13546 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.336050 13521 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:45.346484 13361 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:45.446815 13521 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:45.351902 13574 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:45.445807  8894 raft_consensus.cc:2368] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15.
I20260709 13:01:45.363152 13379 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:45.448547  9442 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:45.452534  8890 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:45.370772 13576 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.442605 13406 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:45.443233 13544 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.465101  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1878) took 1243 ms. Trace:
I20260709 13:01:45.467978  8815 rpcz_store.cc:276] 0709 13:01:44.213412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.276638 (+ 63226us) service_pool.cc:224] Handling call
0709 13:01:45.456088 (+1179450us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b5400083ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:45.456226 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87035136}
I20260709 13:01:45.360311 13435 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
W20260709 13:01:45.492383  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 160314us
I20260709 13:01:45.381162 13561 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.448828 13521 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:45.496038  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 139077us
I20260709 13:01:45.469260 13576 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.453491 13379 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:45.504163  9448 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 13:01:45.470314 13511 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.355374 13597 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.507121  9448 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:45.510423 13372 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.374233 13457 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:45.513911 13596 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:01:45.514413 13599 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:45.447041 13556 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.521421  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:45.502547 13521 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.508416 13379 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.492563 13551 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.567972  9445 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:45.467371 13574 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.525820 13549 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:45.615644  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:45.616374  8740 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:45.561655 13606 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.611763  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.586802 13543 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:45.523634 13404 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:01:45.586396 13475 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:45.632917  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 307811us
I20260709 13:01:45.569697 13429 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.631783 13361 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:45.518952 13511 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.620117 13428 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.630527 13394 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:45.635360 13547 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:01:45.677767  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:45.657465 13544 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.688056 13558 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.688163 13598 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.688794 13598 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.691121 13598 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:45.694260 13540 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.443s	user 0.000s	sys 0.003s
I20260709 13:01:45.729691  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56580 (request call id 1) took 1718 ms. Trace:
I20260709 13:01:45.641786 13543 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.730183  8745 rpcz_store.cc:276] 0709 13:01:43.966436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.966583 (+   147us) service_pool.cc:224] Handling call
0709 13:01:45.664048 (+1697465us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54002df620. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f644c520 00007f90f64c9652 00007f90fb49760a 00007f90f4a1d7d4 00007f90f4a1d762 00007f90f4a1d691 00007f90f4a1d60c 00007f90f4a1c8c2 00007f90f480a48c 00007f90f48098c6
0709 13:01:45.684901 (+ 20853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074084608}
I20260709 13:01:45.742157  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:45.743109  8897 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:45.686245 13544 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:45.637208 13606 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.674160 13394 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.687067 13557 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:01:45.676014 13261 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:01:45.777604 13557 consensus_queue.cc:1243] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:45.778180 13557 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:45.778496 13557 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:45.779299 13557 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.780591 13557 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:45.790819  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 795) took 1358 ms. Trace:
I20260709 13:01:45.791270  8748 rpcz_store.cc:276] 0709 13:01:44.432063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.184016 (+751953us) service_pool.cc:224] Handling call
0709 13:01:45.261548 (+ 77532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.276009 (+ 14461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.276046 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.305794 (+ 29748us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.576083 (+270289us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:45.576132 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.576210 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.790582 (+214372us) raft_consensus.cc:1685] finished
0709 13:01:45.790602 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.790756 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2625,"raft.queue_time_us":527,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":21770,"thread_start_us":222315,"threads_started":1,"wal-append.queue_time_us":291357}
I20260709 13:01:45.793561  8748 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:45.794121  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1896) took 1016 ms. Trace:
I20260709 13:01:45.794332  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1895) took 1095 ms. Trace:
I20260709 13:01:45.794457  8748 rpcz_store.cc:276] 0709 13:01:44.777840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.792880 (+1015040us) service_pool.cc:224] Handling call
0709 13:01:45.793202 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.793904 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.794071 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.795758 13601 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:01:45.794844  8745 rpcz_store.cc:276] 0709 13:01:44.699197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.763167 (+1063970us) service_pool.cc:224] Handling call
0709 13:01:45.793879 (+ 30712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.794048 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.794071 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.794112 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.794143 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.794197 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.794285 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.698030 13406 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:01:45.820303  8821 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:45.775949 13546 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:45.833748 13616 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:45.835093 13618 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:45.835732 13618 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.775771 13544 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.845932 13556 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.846662 13556 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.847193 13556 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.850481 13511 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.851066 13511 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:45.851651 13511 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:45.855986 13497 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.953s	user 0.004s	sys 0.007s
I20260709 13:01:45.856985  8822 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:45.816514 13406 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:45.871662 13551 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.814332 13413 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:45.959164 13413 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:45.975564 13413 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.978283 13407 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.885819 13556 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.983435  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:45.984316  8816 raft_consensus.cc:2368] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:45.984731  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1884) took 1090 ms. Trace:
I20260709 13:01:45.803462 13361 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.985037  8816 rpcz_store.cc:276] 0709 13:01:44.894705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.983128 (+1088423us) service_pool.cc:224] Handling call
0709 13:01:45.984669 (+  1541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.993613 13561 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.993989 13597 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:45.995026 13305 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:45.995277  8819 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:01:46.001086  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1878) took 2020 ms (client timeout 1603 ms). Trace:
I20260709 13:01:45.869141 13576 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:45.993659  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.005892  8823 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
W20260709 13:01:46.001600  8743 rpcz_store.cc:269] 0709 13:01:43.980602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.980723 (+   121us) service_pool.cc:224] Handling call
0709 13:01:46.000824 (+2020101us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54000b6654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:46.001003 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2216000896}
I20260709 13:01:46.014017  8745 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:46.018023  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.028842  8821 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:46.029114  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:46.029539  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.031605  8819 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:46.032251  8823 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:46.030599  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 803) took 1126 ms. Trace:
I20260709 13:01:46.032805  8743 rpcz_store.cc:276] 0709 13:01:44.904430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.007191 (+1102761us) service_pool.cc:224] Handling call
0709 13:01:46.030095 (+ 22904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.030292 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.030319 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:46.030374 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:46.030407 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.030459 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.030551 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.026684 13578 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.036273  8823 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:46.036091 13413 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:46.039201  8743 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:46.015367 13622 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.042570 13618 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.006279 13521 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:46.058753 13583 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.260s	user 0.000s	sys 0.002s
W20260709 13:01:46.046942 13560 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.513s	user 0.000s	sys 0.016s
I20260709 13:01:46.047326  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:46.023334 13546 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.051903  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2167) took 1131 ms. Trace:
I20260709 13:01:46.066622  8743 rpcz_store.cc:276] 0709 13:01:44.920398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.034193 (+1113795us) service_pool.cc:224] Handling call
0709 13:01:46.038918 (+  4725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.051595 (+ 12677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.051841 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.028920 13372 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:46.027338 13511 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.059810 13521 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.053804 13413 consensus_queue.cc:1243] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:46.057615  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.029052 13621 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:46.028723 13620 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.080730  8743 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:01:46.084147  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2168) took 1151 ms. Trace:
I20260709 13:01:46.084486  8743 rpcz_store.cc:276] 0709 13:01:44.933196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.067777 (+1134581us) service_pool.cc:224] Handling call
0709 13:01:46.080495 (+ 12718us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:46.083851 (+  3356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.084093 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.081390  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.082506  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1879) took 1826 ms. Trace:
I20260709 13:01:46.088405  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:46.091235 13628 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.091840 13628 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.094183  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.094429 13618 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.094862  8898 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:46.093963 13372 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.094978 13618 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:46.096448 13612 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:01:46.094228 13379 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
I20260709 13:01:46.097801  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 796) took 1665 ms. Trace:
I20260709 13:01:46.097201 13372 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.097687 13379 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.097158 13556 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:46.098259  8741 rpcz_store.cc:276] 0709 13:01:44.432358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.184307 (+751949us) service_pool.cc:224] Handling call
0709 13:01:45.207879 (+ 23572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.260499 (+ 52620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.260527 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.276960 (+ 16433us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.538965 (+262005us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:45.539020 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.539122 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.097576 (+558454us) raft_consensus.cc:1685] finished
0709 13:01:46.097595 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.097746 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":49680,"thread_start_us":222883,"threads_started":1,"wal-append.queue_time_us":583430}
I20260709 13:01:46.099169 13556 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.099902 13379 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.101588 13556 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:46.104156 13613 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 13:01:46.106526 13379 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.107105 13379 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:46.107743 13605 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.516s	user 0.002s	sys 0.000s
I20260709 13:01:46.109491  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1877) took 1896 ms. Trace:
I20260709 13:01:46.110234  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2443) took 1178 ms. Trace:
I20260709 13:01:46.109479 13379 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.111171  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 802) took 1208 ms. Trace:
I20260709 13:01:46.113674  8745 rpcz_store.cc:276] 0709 13:01:44.902720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.797018 (+894298us) service_pool.cc:224] Handling call
0709 13:01:46.002890 (+205872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.014480 (+ 11590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.110890 (+ 96410us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000b6650. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:46.111076 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397469568}
I20260709 13:01:46.120921  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:46.110749  8891 rpcz_store.cc:276] 0709 13:01:44.931443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.978265 (+ 46822us) service_pool.cc:224] Handling call
0709 13:01:44.978609 (+   344us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:01:45.106120 (+127511us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:45.106141 (+    21us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:45.112050 (+  5909us) log.cc:844] Serialized 78 byte log entry
0709 13:01:45.314947 (+202897us) spinlock_profiling.cc:243] Waited 963 us on lock 0x7b18003fad80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:01:45.524129 (+209182us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:45.524188 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.524282 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.103448 (+579166us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.110034 (+  6586us) raft_consensus.cc:1685] finished
0709 13:01:46.110054 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.110183 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17847,"raft.queue_time_us":23926,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3447,"spinlock_wait_cycles":134614144,"thread_start_us":208951,"threads_started":1,"wal-append.queue_time_us":480791}
I20260709 13:01:46.125880 13390 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:46.126097  8743 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:01:46.126869  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2171) took 1118 ms. Trace:
I20260709 13:01:46.126920  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.127439  8743 rpcz_store.cc:276] 0709 13:01:45.008884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.085186 (+1076302us) service_pool.cc:224] Handling call
0709 13:01:46.125794 (+ 40608us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:46.126571 (+   777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.126815 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.109907  8818 rpcz_store.cc:276] 0709 13:01:44.212836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.212990 (+   154us) service_pool.cc:224] Handling call
0709 13:01:44.242982 (+ 29992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.312684 (+1069702us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b18003c2440. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64c7bfb 00007f90f64d264b
0709 13:01:45.312846 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:45.313192 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.608414 (+295222us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.627577 (+ 19163us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.627717 (+   140us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.627751 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.627816 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.107037 (+479221us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.109266 (+  2229us) raft_consensus.cc:1685] finished
0709 13:01:46.109303 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.109435 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2774,"prepare.queue_time_us":727,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":205343,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":23195,"spinlock_wait_cycles":339017344,"thread_start_us":408633,"threads_started":3,"wal-append.queue_time_us":175393}
I20260709 13:01:46.126799  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.054370 13578 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.133539  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.085896  8820 rpcz_store.cc:276] 0709 13:01:44.252258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.253152 (+   894us) service_pool.cc:224] Handling call
0709 13:01:44.253752 (+   600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.552413 (+298661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.552462 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:44.558321 (+  5859us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.660626 (+102305us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:44.660668 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.660740 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.058131 (+1397391us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.060213 (+  2082us) raft_consensus.cc:1685] finished
0709 13:01:46.060233 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.078806 (+ 18573us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":379,"raft.queue_time_us":112602,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":15941,"spinlock_wait_cycles":5684096,"thread_start_us":334489,"threads_started":2,"wal-append.queue_time_us":174574}
W20260709 13:01:46.141712  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.023s	sys 0.012s
W20260709 13:01:46.142086  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.023s	sys 0.012s
I20260709 13:01:46.142658  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:46.145503  8743 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:46.145654  9442 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:46.146131  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2176) took 1004 ms. Trace:
I20260709 13:01:46.146656  8743 rpcz_store.cc:276] 0709 13:01:45.141976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.128369 (+986393us) service_pool.cc:224] Handling call
0709 13:01:46.138933 (+ 10564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.145889 (+  6956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.146078 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.151264  8745 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:46.075913 13546 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.152714  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1873) took 1949 ms. Trace:
I20260709 13:01:46.121209 13417 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:46.159399  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.156888  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.160458  8820 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:46.153970  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2175) took 1012 ms. Trace:
I20260709 13:01:46.143815  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:46.060494 13622 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.164693  8743 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:01:46.161227  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:46.155252  8824 rpcz_store.cc:276] 0709 13:01:44.203217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.203415 (+   198us) service_pool.cc:224] Handling call
0709 13:01:44.216668 (+ 13253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.220336 (+  3668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.220381 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:44.237877 (+ 17496us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.357605 (+119728us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:44.357665 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.357767 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.046161 (+1688394us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.146985 (+100824us) raft_consensus.cc:1685] finished
0709 13:01:46.147014 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.152642 (+  5628us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2990,"raft.queue_time_us":105857,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":25618,"spinlock_wait_cycles":18782336,"thread_start_us":119241,"threads_started":1,"wal-append.queue_time_us":234192}
I20260709 13:01:46.162994  8891 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:46.177479  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.169632  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.156934 13558 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.161123 13634 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:46.131672 13305 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:01:46.126576 13390 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.179698  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2177) took 1037 ms. Trace:
I20260709 13:01:46.187479  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.115149 13621 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.181021  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.186655  8743 rpcz_store.cc:276] 0709 13:01:45.142266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.147795 (+1005529us) service_pool.cc:224] Handling call
0709 13:01:46.162336 (+ 14541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.175332 (+ 12996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.179644 (+  4312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.182917  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:46.183171  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.122462 13620 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.189280  8816 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.189455  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:46.189704  8743 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:01:46.180760 13556 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.187587  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.190871  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:46.198028  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.183840  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.193849  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:46.200042  8821 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.200830  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.201587  8891 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:46.172850  8745 rpcz_store.cc:276] 0709 13:01:45.141681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.114855 (+973174us) service_pool.cc:224] Handling call
0709 13:01:46.150852 (+ 35997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.151972 (+  1120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.153896 (+  1924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.202049  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2178) took 1048 ms. Trace:
I20260709 13:01:46.214612  8743 rpcz_store.cc:276] 0709 13:01:45.142577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.189157 (+1046580us) service_pool.cc:224] Handling call
0709 13:01:46.189479 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:46.190359 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.190548 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.166244 13417 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.222430  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.186390 13551 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:46.227464  8820 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:46.227149  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:46.191191 13634 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:46.218225 13629 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:01:46.183442 13630 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:01:46.262079  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:46.228737  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1900) took 1072 ms. Trace:
I20260709 13:01:46.199379 13576 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.251744  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:46.227981  8743 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:46.233744 13624 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.421s	user 0.000s	sys 0.003s
I20260709 13:01:46.258653  9449 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:46.296934  9448 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.296415  8745 rpcz_store.cc:276] 0709 13:01:45.156245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.202460 (+1046215us) service_pool.cc:224] Handling call
0709 13:01:46.226679 (+ 24219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.228366 (+  1687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.228654 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.223613 13556 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:46.299511  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122965us
I20260709 13:01:46.303179  8864 leader_election.cc:400] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:46.303670  8864 leader_election.cc:403] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.304675  8816 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:46.311928  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.313155  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.313455  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2181) took 1046 ms. Trace:
I20260709 13:01:46.313983  8891 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:46.313884  8743 rpcz_store.cc:276] 0709 13:01:45.260605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.219310 (+958705us) service_pool.cc:224] Handling call
0709 13:01:46.223625 (+  4315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.306941 (+ 83316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.307143 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.316372  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:46.315594  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 172057us
I20260709 13:01:46.317519  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:46.318327  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96924.8us
I20260709 13:01:46.235560 13644 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:46.322031  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.321708  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.326236  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.326686  8821 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:46.335517  9448 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:46.344519  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.294097 13551 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.386159  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:46.392505 13635 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:01:46.489756  8742 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:46.393652 13633 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:01:46.490713  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.491771  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.492057  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1892) took 1793 ms. Trace:
I20260709 13:01:46.492945  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.492558  8740 rpcz_store.cc:276] 0709 13:01:44.698246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.640138 (+941892us) service_pool.cc:224] Handling call
0709 13:01:45.670879 (+ 30741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.684782 (+ 13903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.685620 (+   838us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.755469 (+ 69849us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.087097 (+331628us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:46.087240 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.087638 (+   398us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.491781 (+404143us) raft_consensus.cc:1685] finished
0709 13:01:46.491809 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.491989 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4653,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":19878,"thread_start_us":264035,"threads_started":1,"wal-append.queue_time_us":397035}
W20260709 13:01:46.402209 13637 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:01:46.494287  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.410207 13634 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.495303  8742 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:46.496187  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.497793  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.411623 13556 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.413010  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1894) took 1714 ms. Trace:
I20260709 13:01:46.498878  8749 rpcz_store.cc:276] 0709 13:01:44.698889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.664865 (+965976us) service_pool.cc:224] Handling call
0709 13:01:45.676686 (+ 11821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.678591 (+  1905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.678633 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.728810 (+ 50177us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.801360 (+ 72550us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:45.801408 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.801499 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.099021 (+297522us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4800080058. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:01:46.347646 (+248625us) raft_consensus.cc:1685] finished
0709 13:01:46.347680 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.412940 (+ 65260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1466,"raft.queue_time_us":7733,"raft.run_cpu_time_us":2571,"raft.run_wall_time_us":401514,"spinlock_wait_cycles":72958720,"thread_start_us":62086,"threads_started":1,"wal-append.queue_time_us":217628}
I20260709 13:01:46.500814  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.501390  8749 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:46.414309 13620 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.322618 13644 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.502765  8749 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:46.503090 13644 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.503336  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2182) took 1162 ms. Trace:
I20260709 13:01:46.503767  8749 rpcz_store.cc:276] 0709 13:01:45.340761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.501952 (+1161191us) service_pool.cc:224] Handling call
0709 13:01:46.502234 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.503119 (+   885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.503290 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.504462  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.415623 13578 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.505771 13578 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:46.506330 13578 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:46.506894 13551 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:46.505028  8742 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:46.507398 13578 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.419198 13390 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.421682  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 805) took 1392 ms. Trace:
I20260709 13:01:46.509819  8741 rpcz_store.cc:276] 0709 13:01:45.028931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.099903 (+1070972us) service_pool.cc:224] Handling call
0709 13:01:46.100231 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.100651 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.100681 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.101172 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.232021 (+130849us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:46.232085 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.232190 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.421437 (+189247us) raft_consensus.cc:1685] finished
0709 13:01:46.421462 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.421618 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":96213,"threads_started":1,"wal-append.queue_time_us":228109}
I20260709 13:01:46.422087 13546 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.423118 13305 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:46.425987  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.512729 13305 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:46.513226 13305 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.515282 13305 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:46.512991  8745 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:46.531716  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 214290us
I20260709 13:01:46.533736  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:46.534631 13650 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.535223 13650 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:46.536175 13650 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.345464  8743 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:46.541623  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.542915  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.543783  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.544791  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.545569  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.546039 13545 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:46.546679 13545 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:46.487504 13622 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.488476  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.549909  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.549969  8898 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:46.550640  8820 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:01:46.532828  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 217796us
W20260709 13:01:46.533919 13623 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.661s	user 0.001s	sys 0.000s
I20260709 13:01:46.540820  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.552423  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:46.485436 13621 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.553192  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.554235 13639 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:46.552106  8818 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:01:46.555397 13574 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:46.556748  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.557775  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.559640  8891 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:46.553377  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:46.483855 13417 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.561488  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:46.554919 13639 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:46.551072  8644 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 199) took 21062 ms (client timeout 15000 ms). Trace:
I20260709 13:01:46.561002 13621 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:46.563262 13393 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:46.564670 13625 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.715s	user 0.000s	sys 0.001s
I20260709 13:01:46.567128 13305 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.567752 13646 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:46.569007  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:46.569645 13651 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:46.572244  8789 leader_election.cc:400] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:46.572588 13618 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.591416 13646 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.595281  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.575742  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.578493  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.580459  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.593914  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:46.598524  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.598373  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.601229  8745 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:46.600484  8891 raft_consensus.cc:2376] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12. Re-sending same reply.
I20260709 13:01:46.590034 13598 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.590476 13417 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.595556 13646 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.594844 13621 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.622223  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.602062 13650 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:46.606129 13618 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:46.700181  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101737us
I20260709 13:01:46.700749  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:46.622291 13546 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.701902  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.625052 13598 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:46.605382 13644 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.618865 13556 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:01:46.705996  8891 raft_consensus.cc:2376] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14. Re-sending same reply.
I20260709 13:01:46.701205  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:46.706874  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:46.657101  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65112.6us
I20260709 13:01:46.618557 13622 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
W20260709 13:01:46.721046  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1886) took 1823 ms (client timeout 1979 ms). Trace:
I20260709 13:01:46.722618 13217 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:46.723043 13217 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:46.721410  8817 rpcz_store.cc:269] 0709 13:01:44.897191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.659165 (+1761974us) service_pool.cc:224] Handling call
0709 13:01:46.720991 (+ 61826us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:46.725371  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2222) took 1790 ms (client timeout 1830 ms). Trace:
I20260709 13:01:46.621977 13361 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:01:46.699433 13630 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.723428 13217 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:46.725878  8894 rpcz_store.cc:269] 0709 13:01:44.929136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.938942 (+  9806us) service_pool.cc:224] Handling call
0709 13:01:46.719052 (+1780110us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400247f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:46.719188 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128390016}
I20260709 13:01:46.651628 13620 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:01:46.702162 13650 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:46.701261 13578 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.625386 13574 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 13:01:46.744745  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 151929us
I20260709 13:01:46.748907  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1889) took 2110 ms. Trace:
I20260709 13:01:46.657774 13634 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:46.762895 13620 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:46.780452  8746 rpcz_store.cc:276] 0709 13:01:44.616853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.450352 (+833499us) service_pool.cc:224] Handling call
0709 13:01:45.450771 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:45.454028 (+  3257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.454072 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:45.474115 (+ 20043us) log.cc:844] Serialized 52 byte log entry
0709 13:01:45.635385 (+161270us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.635483 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.636854 (+  1371us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.564036 (+927182us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.718742 (+154706us) raft_consensus.cc:1685] finished
0709 13:01:46.718767 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.727047 (+  8280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":656,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12719,"spinlock_wait_cycles":66140416,"thread_start_us":112123,"threads_started":1,"wal-append.queue_time_us":320548}
W20260709 13:01:46.772040  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67843.4us
I20260709 13:01:46.777689  8891 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:46.781209  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.728287 13618 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:46.791615  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90603us
W20260709 13:01:46.793751 13658 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:01:46.578537  8644 rpcz_store.cc:269] 0709 13:01:25.557560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.577404 (+ 19844us) service_pool.cc:224] Handling call
0709 13:01:26.046499 (+469095us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6439090 00007f90f64d7f3a 00007f90fc28e1cc 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:01:46.550848 (+20504349us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:41.445641 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:41.446230 (+   589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:41.446266 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:01:41.446278 (+    12us) tablet.cc:662] Decoding operations
0709 13:01:41.447300 (+  1022us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:41.447354 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:41.447384 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:01:41.448158 (+   774us) tablet.cc:701] Row locks acquired
0709 13:01:41.448173 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:01:41.448330 (+   157us) write_op.cc:270] Start()
0709 13:01:41.448443 (+   113us) write_op.cc:276] Timestamp: P: 1783602101448278 usec, L: 0
0709 13:01:41.448461 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:01:41.449121 (+   660us) log.cc:844] Serialized 4047 byte log entry
0709 13:01:41.747723 (+298602us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b
0709 13:01:42.525895 (+778172us) op_driver.cc:464] REPLICATION: finished
0709 13:01:42.600552 (+ 74657us) write_op.cc:301] APPLY: starting
0709 13:01:42.648702 (+ 48150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:42.704578 (+ 55876us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:42.704602 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:43.140862 (+436260us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:43.141250 (+   388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:01:43.153354 (+ 12104us) write_op.cc:312] APPLY: finished
0709 13:01:43.192708 (+ 39354us) log.cc:844] Serialized 97 byte log entry
0709 13:01:43.201213 (+  8505us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:43.206552 (+  5339us) write_op.cc:454] Released schema lock
0709 13:01:43.207222 (+   670us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:43.210038 (+  2816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:43.210076 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":68002,"reported_tablets":35,"spinlock_wait_cycles":317467392,"child_traces":[["op",{"apply.queue_time_us":45844,"apply.run_cpu_time_us":23102,"apply.run_wall_time_us":621768,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45907,"num_ops":10,"prepare.queue_time_us":128825,"prepare.run_cpu_time_us":36196,"prepare.run_wall_time_us":435127,"raft.queue_time_us":144069,"raft.run_cpu_time_us":8548,"raft.run_wall_time_us":27220,"replication_time_us":1077286,"spinlock_wait_cycles":18615424,"thread_start_us":559201,"threads_started":4,"wal-append.queue_time_us":494212,"wal-append.run_cpu_time_us":31278,"wal-append.run_wall_time_us":2283868}]]}
I20260709 13:01:46.798043  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.799333 13664 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:46.801800 13650 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.756630 13630 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.781868 13620 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:46.784170 13618 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:46.827632  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.829006  9449 raft_consensus.cc:2410] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:46.834472  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.837752 13639 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.828059 13417 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:01:46.841485 13598 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.843640 13665 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:01:46.844424 13628 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.846138 13621 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.847681 13644 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:46.790717 13558 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:01:46.853430  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68588.6us
I20260709 13:01:46.809298 13662 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:46.855036 13657 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:01:46.855183 13662 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:46.855818 13662 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.855069  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.858645  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.859462  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:46.863375  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.863586  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.870784  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.871161  9442 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:46.871546  9448 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:46.877501 13558 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:46.877924 13544 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:46.878938 13659 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:46.879323 13606 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:46.880402 13620 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.881901 13620 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.885042  8819 pending_rounds.cc:85] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:01:46.885568  8819 pending_rounds.cc:107] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:01:46.885905  8819 raft_consensus.cc:2889] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:46.888481  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:46.888393  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:46.889154  9442 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.888695 13618 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:46.889276  8820 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:46.891835  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.890825  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.900441  9448 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:46.900563  9443 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:46.902187  9442 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:46.905313  8818 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:46.909142  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.910285  8894 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:46.912526  8894 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:46.892025 13618 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.915561 13646 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.891304 13543 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:46.926210 13217 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:46.931581  8894 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:46.934211  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:46.934878  9449 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:46.926409  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1893) took 1315 ms. Trace:
I20260709 13:01:46.937958  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:46.938552  8818 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.941233  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.935779  8815 rpcz_store.cc:276] 0709 13:01:45.610697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.610815 (+   118us) service_pool.cc:224] Handling call
0709 13:01:45.633315 (+ 22500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.964407 (+331092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.964473 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.007362 (+ 42889us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.172841 (+165479us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:46.172885 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.925861 (+752976us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:46.925929 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.925991 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:46.926011 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.926328 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204366,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12261,"spinlock_wait_cycles":386121472,"thread_start_us":265795,"threads_started":2,"wal-append.queue_time_us":259013}
W20260709 13:01:46.895920 13604 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.253s	user 0.006s	sys 0.001s
I20260709 13:01:46.954706  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.955380  8895 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:46.956344  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.958637  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:46.956935  8892 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:46.959471  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:46.962016  8896 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:46.962658  8897 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 9: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:46.975445  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 802) took 1698 ms. Trace:
I20260709 13:01:46.975978  8893 rpcz_store.cc:276] 0709 13:01:45.276740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.277048 (+   308us) service_pool.cc:224] Handling call
0709 13:01:45.277475 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.301254 (+ 23779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.301705 (+   451us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.309856 (+  8151us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.352957 (+ 43101us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:45.353005 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.353400 (+   395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.841484 (+1488084us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b48001d01d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:01:46.895307 (+ 53823us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:46.975179 (+ 79872us) raft_consensus.cc:1685] finished
0709 13:01:46.975209 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.975372 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9395,"raft.run_cpu_time_us":10743,"raft.run_wall_time_us":1541761,"spinlock_wait_cycles":111121024,"thread_start_us":42662,"threads_started":1,"wal-append.queue_time_us":276942}
I20260709 13:01:46.949182 13217 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:46.983458 13558 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:47.002827 13650 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:47.003567 13650 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.004426 13662 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.005870 13650 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:47.008329  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70865.8us
I20260709 13:01:47.015635  8815 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:01:47.022082 13598 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:47.023257 13598 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.026423 13557 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:47.034566  8818 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:47.036979  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:47.039603 13677 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:47.040184 13677 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:47.047857  8815 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:47.014272 13650 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:47.051369  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1883) took 2162 ms (client timeout 1736 ms). Trace:
W20260709 13:01:47.051985  8822 rpcz_store.cc:269] 0709 13:01:44.889265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.893418 (+  4153us) service_pool.cc:224] Handling call
0709 13:01:47.051057 (+2157639us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540012b954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:47.051243 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207183616}
I20260709 13:01:47.049077 13651 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.047497 13551 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:47.049077 13645 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.041096 13677 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.058815 13651 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.061224 13673 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.070354 13551 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.083681 13511 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.084240 13511 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:47.085325 13511 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.081861 13673 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.094087 13673 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.071731 13645 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.094873 13621 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.077704 13662 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.095314 13621 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:47.060446 13650 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.096228 13621 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.096709 13645 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.097350 13650 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.064545 13557 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:01:47.088768 13622 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:47.100174 13622 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:47.091028 13636 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:47.088792 13651 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.103018  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 801) took 2299 ms. Trace:
I20260709 13:01:47.084957 13551 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:47.103595  8748 rpcz_store.cc:276] 0709 13:01:44.803602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.795917 (+992315us) service_pool.cc:224] Handling call
0709 13:01:45.796202 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.796395 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:45.827029 (+ 30634us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.183778 (+356749us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.101033 (+917255us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b180046d200. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:01:47.101248 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.102622 (+  1374us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:47.102684 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.102791 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.102946 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21186,"prepare.queue_time_us":278196,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":6929,"raft.queue_time_us":2515,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":11211,"spinlock_wait_cycles":3421952,"thread_start_us":383073,"threads_started":2,"wal-append.queue_time_us":434568}
I20260709 13:01:47.105991 13636 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:47.130848  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.152244 13390 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.152465  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.153127  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.153693  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.154241  8819 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.154716  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.155196  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:47.156033  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.157303  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.127058  8816 pending_rounds.cc:85] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 4
I20260709 13:01:47.160672  8816 pending_rounds.cc:107] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:47.161062  8816 raft_consensus.cc:2889] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:47.168202  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.168970  9449 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:47.158226  8824 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:47.175674  9442 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:47.176826  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:47.115988 13621 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.187934  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.127192 13576 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.189440  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.190598  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.191509  8896 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:47.192406  8824 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:47.193682  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:47.195142  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 78008.5us
I20260709 13:01:47.198366  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.153000 13390 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.201731  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.210683  9441 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:47.209127  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.186026 13598 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:47.216735  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.220041  8824 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
W20260709 13:01:47.223688  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53743.9us
I20260709 13:01:47.220597 13546 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.198835 13606 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.234443  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:47.236632  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:47.237447 13429 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.238076 13429 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.240618 13429 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.241451 13379 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:47.241855 13379 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.242213 13379 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.245456 13543 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:47.226781 13437 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:47.247285 13675 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:01:47.221169 13578 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.195230 13576 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.265087 13638 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.265930 13638 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.265084  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.270040 13645 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.271605 13645 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.272100 13645 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.269276 13638 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.277602  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.278533  8748 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:47.257975  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.288856  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.298900 13506 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:47.299518 13506 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.300004 13506 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.307614  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.309171 13572 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:47.309852 13572 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:47.311015 13572 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:47.316234 13685 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:01:47.325800  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:47.330921  8741 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:01:47.229084 13546 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:47.228663 13404 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.336225 13404 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.338776 13404 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.352164  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.353026  9440 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:47.241147 13429 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:01:47.346313 13664 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.359180  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.360039  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:47.255980 13490 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.346750 13546 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.259786 13578 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:47.254607 13606 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:47.260295 13437 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.385363  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:47.355902 13630 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.333109 13413 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:47.369079 13574 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:01:47.371094 13615 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.601s	user 0.008s	sys 0.012s
I20260709 13:01:47.370146 13407 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:47.370662 13390 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.368819 13544 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:47.367519  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.218459 13636 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.374082  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.375193  8892 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:47.368743  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.377501  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.334092 13543 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:01:47.395498  8819 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:47.403317  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.406685  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.414364  8896 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:47.407768  8895 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:47.380347 13578 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.410851  8824 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:01:47.416922  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.418077  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.419113  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.419930  8824 raft_consensus.cc:2410] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:01:47.366606 13490 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.422748  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.423847  8819 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:47.426002  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 48634.1us
I20260709 13:01:47.426488  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.427093  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.432515  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.433706  9440 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:47.435392  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.450899  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.452490  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:47.409209 13630 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.370500 13664 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.477139  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.478627  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.424746 13490 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.482630 13305 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:47.483078 13305 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.483533 13305 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.484112 13606 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.436177 13407 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:47.434962 13368 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.440737 13662 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.442395 13414 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:47.485824 13684 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:01:47.465823 13636 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.478295 13665 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.465616 13630 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.479140 13650 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:47.545431 13689 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:01:47.550604  8744 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:47.548231 13572 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.546183 13407 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.556735  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.496658 13414 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.567481  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.568094  8744 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:47.550969  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.558924  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:47.568960  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.561779  8820 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:01:47.564649  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.565429  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.566162 13506 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.570919  8820 pending_rounds.cc:85] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 8
I20260709 13:01:47.571373  8820 pending_rounds.cc:107] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:01:47.571779  8820 raft_consensus.cc:2889] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:47.572930  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.575680  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.576124  8824 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:47.505959 13630 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.578964  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.569213  8746 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13. Re-sending same reply.
I20260709 13:01:47.580881  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.582142  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:47.558301 13413 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.554715 13543 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.590941  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.597992  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:47.559976 13404 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.605566  8898 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:47.573302 13414 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.606124  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.607661  8745 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:47.606158  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.579973 13665 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.622867 13428 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.617254 13576 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.630467 13664 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.631388 13546 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.634263 13698 raft_consensus.cc:993] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:01:47.634788 13698 raft_consensus.cc:1081] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:01:47.635155 13379 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.635285 13490 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.635996 13665 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:47.636466 13665 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:47.638242 13697 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:01:47.636547  8743 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.642400 13490 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.644368 13305 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.647936 13437 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.648043 13578 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.648878  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.650281  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.651194 13638 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:47.651502  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.651659 13638 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:47.653937 13699 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:47.636447  8748 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:47.668171  8744 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:47.688658  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:47.692505  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 205972us
I20260709 13:01:47.697973  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:47.704596  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:47.732899  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.655408 13506 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.707224  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.574229 13572 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.607362 13413 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.722461 13645 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.620518 13543 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.673844 13428 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:01:47.721472 13546 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:47.732307  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 169163us
I20260709 13:01:47.733453  8898 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:47.734885  8744 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 9.
I20260709 13:01:47.618261 13404 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:47.728722 13644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.013s	sys 0.011s
W20260709 13:01:47.746191  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 293815us
W20260709 13:01:47.744560  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 193716us
I20260709 13:01:47.756958  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.757884  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.752532  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:47.758211  8896 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.758303  8746 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:47.761320  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.761933  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.762133  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:47.742090 13702 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.743335 13698 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.767724  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.765251  8740 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:47.768327  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.770268  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.772998  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.773098 13606 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:47.756135 13644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.013s	sys 0.011s
I20260709 13:01:47.774003 13404 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.774214  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:47.774411 13644 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.775346  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:47.775605 13413 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.777522 13543 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:47.777765 13626 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.932s	user 0.000s	sys 0.022s
I20260709 13:01:47.779031 13572 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.780594 13645 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.780742 13394 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.780894  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1887) took 3177 ms. Trace:
I20260709 13:01:47.781311 13645 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.781764 13645 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.782104 13645 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.783222 13644 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.783442  8741 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:47.784904 13690 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.279s	user 0.000s	sys 0.007s
I20260709 13:01:47.785998 13379 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:47.770813  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 165760us
I20260709 13:01:47.786413 13506 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.786823 13506 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:47.744510 13638 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.787772 13638 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.787693 13506 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.788952 13506 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.765089 13703 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.789799 13703 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.789932  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.790236 13703 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.791213  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.791857  8822 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.792189 13578 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:47.794986  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.795210  8816 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "318340b5bf3b4640972d271299414f94"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37570
I20260709 13:01:47.795710  9440 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:47.795842  8816 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.796281  8816 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.796594  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:47.797120  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.797345  9446 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:47.798048 13379 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.798498 13379 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.798905 13379 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.799507  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:47.800560  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:47.801420 13413 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.801864 13413 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.802234 13413 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.809809  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.768718  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.810776  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.813455  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.775897  8744 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.815807  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.781353  8747 rpcz_store.cc:276] 0709 13:01:44.603053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.334447 (+731394us) service_pool.cc:224] Handling call
0709 13:01:45.334866 (+   419us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:45.351155 (+ 16289us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:45.494751 (+143596us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.667236 (+172485us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.667566 (+   330us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:45.753329 (+ 85763us) log.cc:844] Serialized 78 byte log entry
0709 13:01:45.753495 (+   166us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.753691 (+   196us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.754011 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.754153 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.754221 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.912422 (+1158201us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b18003cd000. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:47.029217 (+116795us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18003cd000. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:47.755209 (+725992us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:47.780646 (+ 25437us) raft_consensus.cc:1685] finished
0709 13:01:47.780667 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.780811 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":454369,"prepare.queue_time_us":1784443,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":118,"raft.queue_time_us":41059,"raft.run_cpu_time_us":679,"raft.run_wall_time_us":23690,"spinlock_wait_cycles":92921472,"thread_start_us":139841,"threads_started":2,"wal-append.queue_time_us":258772}
I20260709 13:01:47.817369  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.830663  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:47.835495  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66855.4us
I20260709 13:01:47.836503  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.826836  8816 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.771384 13546 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.814749 13696 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:01:47.818454 13428 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.846055 13644 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:47.828248 13543 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.818925 13698 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:47.819284 13572 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:47.857357  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.855718  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.841871  8816 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.864377  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.864641  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:47.837383  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65418.6us
I20260709 13:01:47.865621  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.851519  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.838662  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:47.845598 13711 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.858640  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:47.819763 13662 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:47.829907 13677 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.882679  8894 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:47.883047 13598 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:47.840523 13646 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.871309 13551 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.772533 13702 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.882707  8895 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:47.882711  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:47.885936  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.908826  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:47.875413 13305 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.888357  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:47.821976 13664 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:01:47.889394  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:47.884440  8824 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:47.860289 13621 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:01:47.884090 13662 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:47.904183  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65604.2us
I20260709 13:01:47.854723 13557 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:01:47.884747  9448 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:47.872226 13429 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.870419 13644 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:47.881757  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.909566  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:01:47.911346  8892 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
W20260709 13:01:47.925096  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 94586.5us
I20260709 13:01:47.899678 13677 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:47.902041 13598 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:47.926656  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45013.9us
I20260709 13:01:47.941547  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:47.904917 13711 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:47.911082 13652 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.910565 13709 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.923295 13662 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.928140  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.938222  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.893885 13572 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.903144  8744 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:47.953264  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.924741 13557 consensus_queue.cc:1243] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:47.970717 13429 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:47.927495 13620 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:47.972086 13703 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.952499 13574 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:01:47.974175 13698 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:47.927395 13644 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.976079 13414 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.952268 13394 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:01:47.984995  8749 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:47.986747  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.947690  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:47.926016 13535 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.951414  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.918715 13428 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.940310  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:47.952709 13696 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.942235 13598 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:47.962981 13710 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:47.938305 13677 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.966750 13630 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:47.968597 13702 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:47.994822  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:47.990325  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:47.995368  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86659.4us
W20260709 13:01:48.034093  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43936.7us
I20260709 13:01:48.034350  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:48.034927  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:48.036043  8823 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:01:48.015429  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 89600.6us
W20260709 13:01:48.015928  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.993736 13394 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.993780 13428 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.925817 13665 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:47.953743 13711 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:48.025553  8741 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:48.025853  8822 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:47.951083 13544 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.015s	sys 0.016s
I20260709 13:01:47.969146 13407 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:01:48.038213  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:48.037814  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 113078us
I20260709 13:01:48.051944  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.049153  8824 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:48.040544  8816 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:48.038756  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:48.043040  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:48.057631  8820 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:48.056027  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.062642  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.065944  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.067760 13620 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.068401 13572 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.042928 13428 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:48.052610 13544 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.016s	sys 0.016s
I20260709 13:01:48.081555 13544 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:48.082523 13544 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.069625 13574 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:48.084493  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47627.3us
I20260709 13:01:48.044313 13702 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:48.037598 13394 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:48.043859 13696 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:48.107971 13696 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:48.107785 13622 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.108500 13696 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.102985 13702 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.113567  9446 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:48.113776 13702 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:48.118438 13630 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.109067 13622 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.119112  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.107795 13394 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.124047  8742 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:48.125159  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1905) took 1003 ms. Trace:
I20260709 13:01:48.125655  8818 rpcz_store.cc:276] 0709 13:01:47.121769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.121882 (+   113us) service_pool.cc:224] Handling call
0709 13:01:47.122212 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:47.131064 (+  8852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.131093 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:47.790215 (+659122us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b18002eed60. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a
0709 13:01:47.791016 (+   801us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b2c0005b728. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:01:47.791643 (+   627us) log.cc:844] Serialized 52 byte log entry
0709 13:01:48.015322 (+223679us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:48.015363 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.015454 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:48.124917 (+109463us) raft_consensus.cc:1685] finished
0709 13:01:48.124942 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.125074 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1354,"raft.queue_time_us":33994,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":10988,"spinlock_wait_cycles":30743808,"thread_start_us":223281,"threads_started":1,"wal-append.queue_time_us":246620}
I20260709 13:01:48.127647  8898 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:48.128463 13665 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:48.128893 13703 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.130153  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:48.132731 13428 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.103087 13620 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:48.137550 13707 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.141373  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.146715 13620 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:48.146809 13644 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.147699  8895 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:48.142668  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.149437  8898 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:48.142802  8894 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:48.150172 13417 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:48.152812 13413 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.156068 13414 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:48.137894  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 86063.9us
I20260709 13:01:48.159773  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.144178  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.165697  8891 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:48.145344  8821 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:48.073963  8822 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:01:48.077850 13660 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.332s	user 0.008s	sys 0.005s
I20260709 13:01:48.089366  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:48.043417 13718 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:48.090260  8824 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:48.101966  8893 pending_rounds.cc:85] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:01:48.094041 13606 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:48.102350  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.145364  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.142932 13428 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.173732  8890 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:48.173844  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 71638.7us
I20260709 13:01:48.174616  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:48.177747  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:48.059324 13618 raft_consensus.cc:2764] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:48.085070 13574 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.161267  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.162803  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:48.162618 13414 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:48.163980 13620 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:01:48.172751  8893 pending_rounds.cc:107] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:48.185719  8893 raft_consensus.cc:2889] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.173864 13606 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:01:48.189208  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.187314  8818 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:48.168566  8898 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:48.120733 13677 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.186861 13414 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.098366 13711 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:01:48.185880 13636 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.002s
I20260709 13:01:48.206207  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.176368 13413 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.173177 13630 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.183439 13506 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:48.211081  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.172082 13718 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:48.186558 13574 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.224939  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.212025  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.173869 13428 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:48.226315  8891 raft_consensus.cc:2410] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:01:48.144999 13645 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.163295 13379 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.201687 13644 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.202610 13721 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:48.224378  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.207060 13417 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.205389 13414 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.201683 13620 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.230863  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.231637  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.238169  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.137236 13702 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.193113 13394 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.211323 13407 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:48.232129 13621 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:48.229552 13429 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.258610  8895 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:48.255105  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.228768 13718 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.270785  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.276463  8743 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.272112  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.277525  8745 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:48.218564 13652 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:48.241135 13506 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.287659 13506 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.235522 13721 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.281628  8889 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:48.290385  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 127475us
I20260709 13:01:48.237859 13620 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:48.254781 13417 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:48.292084  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.281689  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.309144  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:48.257364 13428 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:48.260564 13407 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.260936 13621 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:48.257937 13557 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:48.303504 13638 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.001s
I20260709 13:01:48.271518 13543 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.327901  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64975.7us
I20260709 13:01:48.333892  8741 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:01:48.357120  8741 pending_rounds.cc:85] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:01:48.357457  8741 pending_rounds.cc:107] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:01:48.357734  8741 raft_consensus.cc:2889] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.276886 13429 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.304557 13721 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.271698 13562 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:01:48.298369 13724 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.282794 13558 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:48.337270 13407 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:48.333755 13544 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.367144  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.372706  8890 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.297084  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.333964 13636 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:48.344620 13621 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:48.386409 13557 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:48.387215 13557 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:48.387619 13678 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:48.388058 13678 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:48.383509 13664 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:01:48.390854 13664 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:48.391220 13664 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:48.401623  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:48.402235 13428 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.403390  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:48.404526  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.362641 13429 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.408758  8743 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:48.413980  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.422919  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:48.423864  8743 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:48.374571 13544 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:48.442931  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.425617  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:48.362080 13543 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.409420 13652 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.458881  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.412652 13636 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:48.419004 13650 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:48.459857  8743 raft_consensus.cc:2368] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:48.421512 13722 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.455381 13618 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:48.464623 13394 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.464839 13618 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.467061 13618 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.467831  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.465272 13394 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.471271 13414 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.392165 13664 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.389034 13678 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.473985 13414 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.474164 13678 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.476009 13733 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.476576 13733 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.478785 13733 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.481068 13379 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.481560  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.482240 13645 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.482719  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.483553  8894 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:48.484503  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.485178 13696 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.445789 13544 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.489956  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.491554  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.492569  8894 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:48.455904 13677 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.469019  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:48.500636  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86743.6us
I20260709 13:01:48.500874  8890 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:48.474026 13664 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.502322  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 812) took 1215 ms. Trace:
I20260709 13:01:48.383977 13724 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:48.502048 13731 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:01:48.421339 13598 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:48.511972  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.506040  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.522903  8748 rpcz_store.cc:276] 0709 13:01:47.286918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.287823 (+   905us) service_pool.cc:224] Handling call
0709 13:01:47.288210 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.689924 (+401714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.502091 (+812167us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540013c4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:48.502256 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800587008}
I20260709 13:01:48.523154  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.520625  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:48.418149 13620 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.455662 13417 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.538436 13543 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.519410 13652 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:48.541575 13636 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.542647 13598 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.547955  8823 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:48.527232  8890 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:48.495541 13662 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.529018  8743 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:48.522905 13702 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:48.551874  8743 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:48.499369 13665 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:01:48.530773  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:48.503458 13696 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.532315  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:48.533244 13724 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.563460  8816 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:48.535761  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:48.550963  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45023.9us
W20260709 13:01:48.552103  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.557977  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:48.559142  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.552644  8743 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.565723  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.572291  8891 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.584879  8642 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 11, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 11 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:48.587338  8642 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:48.555526 13618 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.555446 13722 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:48.552037 13634 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.565151 13419 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.634987  8748 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
W20260709 13:01:48.636590 13689 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:01:48.638985  8891 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:48.590049 13646 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:48.592840 13703 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:01:48.557940 13652 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.632066 13702 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:48.652251 13419 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:48.658686 13646 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:48.646724 13618 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.646781 13722 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:48.706528  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 116434us
I20260709 13:01:48.724009 13662 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.754591 13718 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.756336  8824 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:48.758531 13618 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.728708 13646 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.766294  8642 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 11, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 11 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.739447 13574 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.832993 13722 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:48.723554 13621 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.839689  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2552) took 1010 ms. Trace:
I20260709 13:01:48.840210  8820 rpcz_store.cc:276] 0709 13:01:47.826515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.024795 (+198280us) service_pool.cc:224] Handling call
0709 13:01:48.836286 (+811491us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b54001b8854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:48.836424 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16231296}
I20260709 13:01:48.723323 13419 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:48.838438 13746 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:01:48.849792  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 160296us
I20260709 13:01:48.784914 13429 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.859032 13574 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:48.859781 13728 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.544s	user 0.002s	sys 0.000s
I20260709 13:01:48.861405  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1917) took 1291 ms. Trace:
I20260709 13:01:48.845927 13718 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.861791  8744 rpcz_store.cc:276] 0709 13:01:47.570318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.992112 (+421794us) service_pool.cc:224] Handling call
0709 13:01:47.992539 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.016503 (+ 23964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.016556 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.017207 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.168911 (+151704us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:48.169065 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.169129 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:48.859211 (+690082us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:48.860992 (+  1781us) raft_consensus.cc:1685] finished
0709 13:01:48.861029 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.861328 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72798,"raft.queue_time_us":28402,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":9455872,"thread_start_us":129321,"threads_started":1,"wal-append.queue_time_us":230163}
I20260709 13:01:48.869937 13511 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.879078 13429 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.883522  8743 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:48.889889  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1919) took 1318 ms (client timeout 1660 ms). Trace:
W20260709 13:01:48.890348  8746 rpcz_store.cc:269] 0709 13:01:47.571325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.580632 (+  9307us) service_pool.cc:224] Handling call
0709 13:01:48.889668 (+1309036us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54004650d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:48.889830 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405967104}
I20260709 13:01:48.894205 13663 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:48.894742 13663 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:48.895674 13663 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.896958 13663 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:48.897719 13621 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.900091  8743 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:48.903098 13261 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.903563 13261 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:48.904408 13261 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.913518 13506 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.875185 13544 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:48.914485  8642 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:01:48.916723  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.916227  8642 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 10, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:48.917467 13544 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:48.909852 13407 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:48.914485 13721 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 13:01:48.919685 13261 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.927186  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.934989  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.935318  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:48.926365  9440 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:48.925159  9446 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:01:48.930285 13598 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.937835 13721 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.945564 13598 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:48.946760  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.946966 13721 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.947620 13598 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:48.948939 13598 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.928951 13743 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:48.950896 13744 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:48.933576  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.936442  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.936720  9449 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:48.926882 13562 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.953724  8893 raft_consensus.cc:2368] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12.
I20260709 13:01:48.942839 13407 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:48.951402 13744 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.940095  8642 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 2 to 9, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:48.955971  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:48.957876  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 392276us
I20260709 13:01:48.959843 13737 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:48.960734 13574 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:48.962528  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 549057us
I20260709 13:01:48.927342 13696 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.966346  9442 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:48.966754  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.962064 13562 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.963104 13394 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.975767  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.977805  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.978076  9443 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:48.976236  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.983099  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.964934 13558 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.987001  9449 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.968328 13739 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.996620  9447 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:48.996582  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:48.998663  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.998860  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:48.964020 13598 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.001616  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.003050  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.003541  9443 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:49.004660  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.011938  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1913) took 1452 ms. Trace:
I20260709 13:01:49.012537  8749 rpcz_store.cc:276] 0709 13:01:47.559104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.575048 (+ 15944us) service_pool.cc:224] Handling call
0709 13:01:49.011677 (+1436629us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001d9554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:49.011857 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931377024}
I20260709 13:01:49.022596  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.989140 13394 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.022667  8889 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260709 13:01:49.030234  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.026479  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:48.999466 13511 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:48.981702 13696 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:48.981741 13574 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.038497  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.039719  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:48.992913 13558 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.034440  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.031391  8895 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:49.000777 13618 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:01:49.052165  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:49.052958 13511 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:49.053467  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.047003  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1914) took 1850 ms. Trace:
I20260709 13:01:49.055020  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:49.055665  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.054711  8819 rpcz_store.cc:276] 0709 13:01:47.196601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.479644 (+283043us) service_pool.cc:224] Handling call
0709 13:01:47.495492 (+ 15848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.495673 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:47.508342 (+ 12669us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.367950 (+859608us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b18003b9560. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:01:49.046682 (+678732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.046732 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:49.046774 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.046847 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.046954 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":390988,"prepare.queue_time_us":12526,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3899,"raft.queue_time_us":359396,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":8828,"spinlock_wait_cycles":19945472,"thread_start_us":325507,"threads_started":2,"wal-append.queue_time_us":1144034}
I20260709 13:01:49.056412  8818 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:49.047078  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.047598 13622 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:49.048527 13544 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:49.047150 13767 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:49.054118 13620 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.054131 13618 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:49.063373  9445 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:49.065343  9443 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:49.070288  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:48.981726 13737 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.072628  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.077255  8898 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:49.072944  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 86914.4us
I20260709 13:01:49.073836  9442 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:48.998179 13739 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.092590  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.075222 13618 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.107054  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2251) took 1243 ms. Trace:
I20260709 13:01:49.107218  9449 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:49.111002  9446 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.107465  8892 rpcz_store.cc:276] 0709 13:01:47.863336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.865208 (+  1872us) service_pool.cc:224] Handling call
0709 13:01:47.882650 (+ 17442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.925521 (+ 42871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.106834 (+1181313us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:49.106995 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64580864}
I20260709 13:01:49.110936  8898 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:49.110139  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:49.092990 13636 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:49.046074 13743 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:01:49.091481 13766 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:49.144747  8816 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:01:49.131927 13769 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.135339  8819 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:01:49.135962  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.137717 13718 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:49.137838  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:49.157379  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.141958 13743 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:49.142660  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:49.142786  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 845921us
I20260709 13:01:49.142957 13720 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:49.142786  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.143672 13562 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.117584  8898 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:49.100683 13630 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:01:49.174619  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.113144 13544 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:01:49.149364  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:49.163129  8818 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:49.116506 13636 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.158025  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:49.168937  8898 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:49.173480  8743 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:49.161892 13718 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:49.163784 13767 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.160931 13743 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.194568  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.205859  8749 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
W20260709 13:01:49.207855  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.184204 13558 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.216954  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:49.221761  8742 raft_consensus.cc:2368] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12.
I20260709 13:01:49.199906 13769 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:49.191136 13766 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:49.226231  8815 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:49.201038 13718 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:49.245637  8820 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:49.237653  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.247444  9444 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:49.238734  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 64268.4us
I20260709 13:01:49.241039  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.241720  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.183645 13677 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:01:49.249893  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.248911  8748 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:49.253475  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.255477  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.255785  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.256270  8743 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:49.224496 13598 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.261741  8746 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:49.261183  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.258005  8745 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:49.267045  8748 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:49.262830  8815 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:01:49.267623  8815 pending_rounds.cc:85] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 9
I20260709 13:01:49.267874  8815 pending_rounds.cc:107] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:49.268110  8815 raft_consensus.cc:2889] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:49.245471 13414 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:49.272564  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.217593 13379 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:49.252259 13652 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:49.252518 13506 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:49.201964 13618 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.249538 13766 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.250528 13709 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:49.263164 13404 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:49.285779  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:49.247889 13407 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:49.291520  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.259769  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.294159 13652 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:49.310233 13652 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.236624 13769 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.315478 13769 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.316887  9446 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:49.289045 13414 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.284567 13390 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.290735 13437 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:49.333665 13414 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.307333 13574 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:49.308857 13768 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:01:49.296658  9440 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:49.283170 13429 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:49.336359 13429 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.249368 13636 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.340667 13390 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:49.320650 13618 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.298764 13720 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:49.354241 13779 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:01:49.373617  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 82241us
I20260709 13:01:49.298561 13743 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.425880  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.426688 13576 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:49.427218  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:49.427748  8822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 1924) took 1500 ms (client timeout 1655 ms). Trace:
I20260709 13:01:49.428090 13576 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:49.428162  8822 rpcz_store.cc:269] 0709 13:01:47.927582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.951220 (+ 23638us) service_pool.cc:224] Handling call
0709 13:01:49.427562 (+1476342us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b540029a4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:49.427691 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78403840}
I20260709 13:01:49.428786  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.429543  8821 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:49.430132 13743 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.433542 13394 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.434235 13576 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.435915  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:49.436697 13576 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:49.437170 13576 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.437364  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2471) took 1221 ms. Trace:
I20260709 13:01:49.437717  8894 rpcz_store.cc:276] 0709 13:01:48.216125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.493188 (+277063us) service_pool.cc:224] Handling call
0709 13:01:48.497255 (+  4067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.524155 (+ 26900us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:48.694556 (+170401us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.436053 (+741497us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b1800372460. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:01:49.436256 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.436429 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.436489 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:49.436548 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.437149 (+   601us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b5400047ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:49.437186 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.437314 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9094,"prepare.queue_time_us":185905,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1947,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5737,"spinlock_wait_cycles":73014784,"thread_start_us":287073,"threads_started":2,"wal-append.queue_time_us":165517}
I20260709 13:01:49.373029 13636 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:49.381861 13780 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:01:49.383927 13747 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.685s	user 0.002s	sys 0.001s
I20260709 13:01:49.454596  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:49.455209  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.455860  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.456657 13744 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.458876  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:49.459129  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.459731 13696 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.460227  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:49.460796  8820 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:49.463202  8817 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:49.464195 13511 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:49.464813 13765 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:49.465775  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.466177  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.466982  8822 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:49.468008  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:49.468283  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.468710  8821 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:49.470672  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2196) took 1892 ms (client timeout 1564 ms). Trace:
I20260709 13:01:49.470890  8898 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:49.470780 13719 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:49.471105  8740 rpcz_store.cc:269] 0709 13:01:47.578113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.578583 (+   470us) service_pool.cc:224] Handling call
0709 13:01:49.470479 (+1891896us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400021cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:49.470610 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343025536}
I20260709 13:01:49.474920  9445 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:49.475399 13379 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:49.385769  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 821) took 1819 ms. Trace:
I20260709 13:01:49.476540 13723 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:49.476943 13723 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.477869  9447 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:49.477766 13723 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.476141  8896 rpcz_store.cc:276] 0709 13:01:47.566016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.575271 (+  9255us) service_pool.cc:224] Handling call
0709 13:01:47.606172 (+ 30901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.758679 (+152507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.383156 (+1624477us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:49.385702 (+  2546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349307264}
I20260709 13:01:49.382397 13737 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.460279 13696 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:49.482266 13696 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.482735 13696 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.483345 13428 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:49.484306 13428 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:49.484658 13428 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.485436 13428 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:49.394250 13749 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.698s	user 0.000s	sys 0.001s
I20260709 13:01:49.338621 13707 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:01:49.487967  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.488709  9442 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:49.400786 13620 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:49.394037 13429 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.410007 13622 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:49.489344 13620 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.489701 13622 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:49.412436  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.433458  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.438917 13698 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:49.492048 13698 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:49.492413 13698 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.493152 13698 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.493950 13765 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.494415 13698 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.494863 13765 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.408747  8745 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:49.440316 13678 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:49.268036  8642 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.497148 13678 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:49.497534 13678 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:49.442342  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.445609  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 260189us
W20260709 13:01:49.499115  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.498299  8642 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 9, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.443544  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2480) took 1215 ms. Trace:
I20260709 13:01:49.490442 13620 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.447096  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2472) took 1230 ms. Trace:
I20260709 13:01:49.501143 13620 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:49.501987  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.460848  8815 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:49.502650 13428 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.384186 13766 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.481017 13720 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:49.504549 13429 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:49.504956 13429 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.438002 13576 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.505710 13429 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.507195 13429 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.498311 13678 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.442754  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.518505  8746 raft_consensus.cc:2368] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:49.521988  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:49.528234  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62445.4us
I20260709 13:01:49.440204  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.526268  8745 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:49.529994 13763 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:49.529057 13744 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:49.527434  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.524101  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:49.530604  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 54660.8us
I20260709 13:01:49.533614  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.342550 13739 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.500537  8890 rpcz_store.cc:276] 0709 13:01:48.228176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.577440 (+349264us) service_pool.cc:224] Handling call
0709 13:01:48.633852 (+ 56412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:48.644527 (+ 10675us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:48.716470 (+ 71943us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.850134 (+133664us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:48.853364 (+  3230us) log.cc:844] Serialized 52 byte log entry
0709 13:01:48.853565 (+   201us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:49.443010 (+589445us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.443244 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.443316 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.443357 (+    41us) raft_consensus.cc:1685] finished
0709 13:01:49.443368 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.443486 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":589187,"prepare.queue_time_us":667377,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":2010,"raft.queue_time_us":11561,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":9306,"thread_start_us":182595,"threads_started":2,"wal-append.queue_time_us":147144}
W20260709 13:01:49.536281  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96161.1us
I20260709 13:01:49.528599  8749 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:49.508460  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:49.538532  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 56457.6us
I20260709 13:01:49.540067  8822 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:49.542932  9412 leader_election.cc:400] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12.
I20260709 13:01:49.543491  9412 leader_election.cc:403] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.517690 13723 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.531224 13437 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:49.566309  8745 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:49.559922  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:49.552194  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 404466us
I20260709 13:01:49.563267  9414 leader_election.cc:400] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12.
W20260709 13:01:49.567902  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59553.6us
I20260709 13:01:49.568214  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.569187  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:49.569737  9446 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:49.570709  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.571324  9440 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
W20260709 13:01:49.571811  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.501490  8897 rpcz_store.cc:276] 0709 13:01:48.216441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.498950 (+282509us) service_pool.cc:224] Handling call
0709 13:01:48.524625 (+ 25675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.524843 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:48.696460 (+171617us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.139954 (+443494us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18003c8d40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:01:49.445933 (+305979us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.446264 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.446566 (+   302us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.446727 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.446769 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:49.446812 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.446935 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.447038 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341863,"prepare.queue_time_us":519820,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":118,"raft.queue_time_us":1902,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":261,"spinlock_wait_cycles":4044672,"thread_start_us":145926,"threads_started":2,"wal-append.queue_time_us":44429}
I20260709 13:01:49.572873  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.573452  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.573856  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.578707  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.582845  9440 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:01:49.581691  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:49.588572  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 125197us
I20260709 13:01:49.592826  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.593784  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.595034  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.598461  9441 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:49.600893  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.531402 13511 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:49.602865  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.603767  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.605580 13719 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.606146 13719 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.608355 13719 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.611673  8749 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:49.612875  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.613143  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:49.614373  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:49.614429  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:49.615454 13783 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:01:49.616930 13574 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:49.589741 13390 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:49.612911  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:49.623469  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.628751  8863 leader_election.cc:400] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:49.629242  8863 leader_election.cc:403] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.623799  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:49.560211 13379 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.675715  8897 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:49.585175 13652 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:49.575387 13739 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12.
I20260709 13:01:49.561648 13407 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:49.631446  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53394.5us
I20260709 13:01:49.709506  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.632066  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.710881  8892 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:49.668249  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.670233  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.625651 13574 consensus_queue.cc:1243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:49.576944 13677 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
W20260709 13:01:49.692324  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.693456  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 584873us
I20260709 13:01:49.736455  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.500124  8642 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:49.738246  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.738461  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.707178  8743 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:49.739406  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.707921  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.739981  8745 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:49.622064 13511 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:49.579571 13678 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:49.739411  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.592597 13733 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:49.590320 13769 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:49.618291 13437 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:49.790050 13437 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.691504  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:49.626145 13544 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:49.631006 13535 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:49.734063  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 141332us
I20260709 13:01:49.795724  8789 leader_election.cc:400] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12.
I20260709 13:01:49.583628 13707 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:49.797571 13707 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.674549 13673 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:01:49.799623 13707 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.763178 13783 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.800981 13783 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.802212  8818 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:49.803050 13783 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:49.803279  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.684053 13664 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:01:49.803856  8816 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:49.804086 13598 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:49.674013 13390 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:49.693033 13379 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.755203  8890 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:49.784463  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.807742  8815 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:49.784627  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.804618 13598 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:49.808255 13379 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:49.806531  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:49.725735 13787 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:01:49.583165 13621 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.127s
W20260709 13:01:49.746493 13728 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:01:49.832979  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.834287  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.591738 13718 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.810916  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:49.848507  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.852133  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.869831  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:49.849404  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:49.743781 13606 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:49.778394 13645 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:49.771126 13652 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:01:49.908998  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59695.3us
W20260709 13:01:49.849327 13755 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:01:49.626379 13722 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.154s
W20260709 13:01:49.899860  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44725.9us
I20260709 13:01:49.932828  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:49.879419 13796 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.004s
W20260709 13:01:49.936743  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.811380 13544 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.908507 13621 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:49.806538 13390 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:49.908249  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 75380.1us
I20260709 13:01:49.989876  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.994081  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:49.903918  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 221440us
I20260709 13:01:49.954149  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:49.954885  8819 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:49.721599 13414 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.135s
I20260709 13:01:49.841948 13783 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.841815 13535 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:49.845732 13707 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:49.848343 13767 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:49.964674  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 255287us
I20260709 13:01:49.999612  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.941166 13677 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:50.015563  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:49.809206 13598 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:49.974800 13511 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.019559 13636 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.019985  8642 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 7, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 7 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.010187 13394 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:50.089872 13678 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:50.089491  8642 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 9, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:01:50.090898  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 91391.7us
W20260709 13:01:50.092535  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77119.5us
I20260709 13:01:49.986445 13544 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.093339  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:50.094323  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:50.095463  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:49.836513 13769 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.092085  8642 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 9, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.982355 13724 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:50.104241  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 480595us
W20260709 13:01:50.105062  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.961730 13766 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.110790  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:50.067229 13799 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:01:49.992110 13652 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:50.085543 13801 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:01:50.067940 13798 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.324s	user 0.000s	sys 0.000s
I20260709 13:01:50.006516 13414 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:01:50.249228  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 138569us
I20260709 13:01:50.273032  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:50.274998  8862 leader_election.cc:400] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:50.275717  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:50.072283 13718 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.096287 13677 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.006772 13415 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.245s
I20260709 13:01:50.019655 13664 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.242s
I20260709 13:01:50.269241 13678 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.269821 13753 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:50.103428 13707 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:50.246136 13636 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:50.259783 13755 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:01:50.265604 13766 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:50.320399  8742 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.005s	sys 0.048s
W20260709 13:01:50.264714 13796 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:01:50.005265 13634 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.260s
I20260709 13:01:50.247083 13739 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:01:50.341037  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.025405 13783 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.085865 13535 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.185830 13394 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:50.294137  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.256600 13511 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.281252 13709 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:01:50.310217 13645 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:01:50.316924  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.310845 13702 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.317408 13707 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.318382 13678 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.293840 13652 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.262298 13598 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.252125 13767 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:01:50.322592  8742 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.005s	sys 0.048s
I20260709 13:01:50.341217  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:50.250979  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 148438us
I20260709 13:01:50.347669  8890 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:50.287860 13677 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:50.294411 13646 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:50.353188  8744 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:50.353511  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.015s	sys 0.044s
W20260709 13:01:50.355190  8741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.090s	sys 0.221s
I20260709 13:01:50.338769 13766 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.367422  8743 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
W20260709 13:01:50.370564  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.015s	sys 0.044s
W20260709 13:01:50.370738  8741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.090s	sys 0.221s
W20260709 13:01:50.374886  9447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.033s	sys 0.067s
W20260709 13:01:50.376816  9447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.033s	sys 0.067s
W20260709 13:01:50.374097  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 745816us
I20260709 13:01:50.370002  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:50.378239  8864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.025s	sys 0.040s
W20260709 13:01:50.417264  8864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.034s	sys 0.043s
I20260709 13:01:50.381798 13823 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:50.417865  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:50.380160  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:50.424762  9443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "011b370821004185a003be210a525c01"
dest_uuid: "112105225cc140459688be5d31dcb071"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:33086
I20260709 13:01:50.427026  9443 raft_consensus.cc:606] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:50.418637  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 792454us
I20260709 13:01:50.364629 13645 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.429845  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.431882  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.433938  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.435940  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.370033 13673 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.376617 13766 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.442595  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:50.443674  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 75026.5us
I20260709 13:01:50.445127  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:50.441311  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:50.448177 13811 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:50.439159 13645 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.448272 13804 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.424764 13437 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:50.451329  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:50.270499  8642 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 5 to 9, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.425633 13823 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:50.495175 13814 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:01:50.452056 13390 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:01:50.503301  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 52091.5us
I20260709 13:01:50.245545 13724 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:50.508098 13769 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:50.514672 13812 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:50.515590 13812 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.521826 13807 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:50.522675 13437 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.523554 13802 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:01:50.523367 13437 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:50.515049 13793 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:01:50.530807 13429 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.023s	user 0.008s	sys 0.007s
I20260709 13:01:50.447255 13673 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.539831  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:50.547631  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 164915us
I20260709 13:01:50.548569  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:50.499181 13808 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:50.551937  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:50.499541 13811 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:50.502672 13678 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:50.522282 13379 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:50.536044 13724 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:50.573033  8894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.045s	sys 0.098s
W20260709 13:01:50.574445  8894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.045s	sys 0.098s
W20260709 13:01:50.581164  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.015s	sys 0.011s
W20260709 13:01:50.581579  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.015s	sys 0.011s
W20260709 13:01:50.571540 13407 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.020s	sys 0.078s
I20260709 13:01:50.478814 13804 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:50.582530  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.555382 13808 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:50.564293 13429 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.565146 13390 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:50.553705 13417 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:50.580873 13576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.025s
W20260709 13:01:50.584693  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.574788 13678 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:50.565332 13722 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:01:50.606489  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:50.607159  9415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.005s	sys 0.025s
I20260709 13:01:50.583070 13769 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.586800 13407 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.020s	sys 0.078s
W20260709 13:01:50.604946 13622 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.115s	user 0.016s	sys 0.000s
I20260709 13:01:50.604611 13379 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:50.608872 13783 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:50.642445  9415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.005s	sys 0.025s
W20260709 13:01:50.648207  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 703863us
W20260709 13:01:50.606257 13576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.001s	sys 0.026s
I20260709 13:01:50.618268 13437 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.606029 13417 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.649907 13646 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:50.652710  8749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.000s	sys 0.016s
I20260709 13:01:50.620747 13407 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:50.635515 13706 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:50.616097 13678 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.624475 13769 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:50.660743  8861 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.007s	sys 0.011s
W20260709 13:01:50.669142  8749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.000s	sys 0.016s
W20260709 13:01:50.680953  8861 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.007s	sys 0.011s
W20260709 13:01:50.643019  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.016s	sys 0.017s
I20260709 13:01:50.632926 13622 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:50.681730  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.016s	sys 0.017s
W20260709 13:01:50.684943  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 589611us
I20260709 13:01:50.648811 13379 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.650341 13836 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:01:50.712007 13678 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:50.713855 13769 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:50.655136 13808 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.687613 13815 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:01:50.715829 13428 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.213s	user 0.011s	sys 0.004s
I20260709 13:01:50.715952 13815 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:50.718357 13698 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.224s	user 0.019s	sys 0.000s
I20260709 13:01:50.720709 13379 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.723272 13417 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.724063 13417 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:50.604118 13823 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.727679  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.728346  8894 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:50.729350 13808 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.724128  8746 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:50.731551  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 840) took 1049 ms. Trace:
I20260709 13:01:50.732493  8817 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:50.701020  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:50.725169 13723 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.159s	user 0.006s	sys 0.004s
I20260709 13:01:50.782358  8816 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:50.739131 13825 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:50.716869 13815 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:50.744083 13817 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:01:50.743714 13646 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.744330 13598 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:50.744587 13818 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:01:50.739131 13739 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:01:50.739385 13769 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.747346 13719 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.748984 13379 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.750000 13698 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:50.756681 13795 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.814s	user 0.000s	sys 0.002s
I20260709 13:01:50.769934  8823 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:50.771978 13621 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.774730 13620 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.273s	user 0.011s	sys 0.004s
I20260709 13:01:50.778412  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.765036  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 812) took 1625 ms. Trace:
I20260709 13:01:50.782226  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:50.687840 13720 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:50.724867 13417 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:50.813669  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 251340us
W20260709 13:01:50.829555  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 128637us
W20260709 13:01:50.791872 13544 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.004s
I20260709 13:01:50.731954  8749 rpcz_store.cc:276] 0709 13:01:49.682240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.708684 (+ 26444us) service_pool.cc:224] Handling call
0709 13:01:49.771610 (+ 62926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.772026 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.772066 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.772630 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.681247 (+908617us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:50.681320 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.681432 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.731208 (+ 49776us) raft_consensus.cc:1685] finished
0709 13:01:50.731228 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.731494 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115742,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":32501,"thread_start_us":908236,"threads_started":1,"wal-append.queue_time_us":932094}
I20260709 13:01:50.829514 13824 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:50.794580 13673 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:50.841334  8818 raft_consensus.cc:2376] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13. Re-sending same reply.
I20260709 13:01:50.840968  8819 rpcz_store.cc:276] 0709 13:01:49.139085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.159304 (+ 20219us) service_pool.cc:224] Handling call
0709 13:01:49.159664 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.995843 (+836179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.764641 (+768798us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54000834d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:50.764942 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828606336}
W20260709 13:01:50.845541  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 272812us
W20260709 13:01:50.845124  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 188816us
W20260709 13:01:50.850418  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.857223  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.852133  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.852495  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.853250  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.802907 13825 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.845326 13574 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.865907 13574 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.867411 13417 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:50.868081 13511 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
I20260709 13:01:50.849795 13753 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:50.879212 13825 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.882644 13733 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:50.883464 13733 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.807005 13407 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.886816  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1936) took 1479 ms. Trace:
I20260709 13:01:50.887640  8742 rpcz_store.cc:276] 0709 13:01:49.407001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.448942 (+ 41941us) service_pool.cc:224] Handling call
0709 13:01:49.449323 (+   381us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:49.449793 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:49.450062 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.637680 (+187618us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:49.684576 (+ 46896us) log.cc:844] Serialized 78 byte log entry
0709 13:01:49.684729 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:49.703651 (+ 18922us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.361423 (+657772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.361521 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.751921 (+390400us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4800080058. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:01:50.755732 (+  3811us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:50.886500 (+130768us) raft_consensus.cc:1685] finished
0709 13:01:50.886531 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.886737 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41599,"prepare.queue_time_us":112749,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":124350,"raft.run_cpu_time_us":12095,"raft.run_wall_time_us":1093584,"spinlock_wait_cycles":71707904,"thread_start_us":795413,"threads_started":2,"wal-append.queue_time_us":275714}
I20260709 13:01:50.849808 13723 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.838795 13769 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:50.908793 13803 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:01:50.869223 13753 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.819326 13719 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.889438 13783 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.839476 13698 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:50.939453  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 205571us
I20260709 13:01:50.839174 13379 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.910693 13428 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.843616 13621 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:50.961380 13379 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:50.926321 13618 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.556s	user 0.008s	sys 0.007s
I20260709 13:01:50.963672 13621 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:50.964246 13621 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:50.889070 13804 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:50.907860 13417 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.975409  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.957873 13813 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:50.968488 13743 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.538s	user 0.002s	sys 0.012s
I20260709 13:01:50.958963 13719 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:50.928180 13696 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.439s	user 0.007s	sys 0.007s
I20260709 13:01:50.989710  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 832) took 1407 ms. Trace:
I20260709 13:01:50.990244  8748 rpcz_store.cc:276] 0709 13:01:49.582184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.582824 (+   640us) service_pool.cc:224] Handling call
0709 13:01:49.609982 (+ 27158us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:49.675955 (+ 65973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.675987 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:49.717761 (+ 41774us) log.cc:844] Serialized 52 byte log entry
0709 13:01:50.378762 (+661001us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:50.415825 (+ 37063us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.416839 (+  1014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.416939 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.989448 (+572509us) raft_consensus.cc:1685] finished
0709 13:01:50.989472 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.989647 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77974,"prepare.queue_time_us":45507,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":10591,"raft.queue_time_us":88731,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":32014,"thread_start_us":445197,"threads_started":1,"wal-append.queue_time_us":792383}
I20260709 13:01:50.992074  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.992676  8819 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:50.971406  8897 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:50.969642  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:50.972366  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.993862  8824 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:50.858937 13544 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:50.857797 13812 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:50.985196  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:50.987565  8742 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:50.988186  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:50.988816  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.014762  8818 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:51.004770  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:51.019546  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.927695 13723 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.028551  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:50.988389 13766 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.004s
I20260709 13:01:50.976715 13428 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.023955 13620 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.015630  8749 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:50.977824 13744 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.965219 13783 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:51.033550  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 124652us
I20260709 13:01:50.995049 13813 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:51.017863 13803 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:51.046927 13766 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.014665 13622 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.018940 13824 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.997099 13630 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.996794 13511 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:51.051853  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.023381 13838 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.056248  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:51.027658 13618 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:50.983035 13417 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:50.994824 13696 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.054154 13652 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.067504 13707 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 13:01:51.060137 13843 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.050511 13743 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:51.089617  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.109848  8819 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:51.079551 13511 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.092670 13622 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.071877 13678 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.066771 13810 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.069458 13698 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.051040 13620 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:51.116791  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.617s (SENT) [suppressed 23 similar messages]
I20260709 13:01:51.080780 13739 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:51.053654 13744 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.076239 13630 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.078706 13673 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:51.135953  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 828) took 1964 ms. Trace:
I20260709 13:01:51.074663 13824 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.143047 13849 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.143363 13851 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:01:51.143782 13849 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.145515 13606 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.146108 13606 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.101315 13618 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.137145  8741 rpcz_store.cc:276] 0709 13:01:49.171884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.171986 (+   102us) service_pool.cc:224] Handling call
0709 13:01:49.172273 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:49.238012 (+ 65739us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:49.439049 (+201037us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b180031c180. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:01:49.457849 (+ 18800us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b180031c180. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:49.589990 (+132141us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.404037 (+814047us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:50.546621 (+142584us) log.cc:844] Serialized 52 byte log entry
0709 13:01:50.546825 (+   204us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:50.546877 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.546989 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.135726 (+588737us) raft_consensus.cc:1685] finished
0709 13:01:51.135746 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.135894 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67711,"prepare.queue_time_us":12637,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":30,"raft.queue_time_us":66365,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":31247,"spinlock_wait_cycles":19135744,"thread_start_us":907491,"threads_started":2,"wal-append.queue_time_us":971424}
I20260709 13:01:51.082239 13838 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:51.178109  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.110769 13652 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.187741 13417 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.187954 13853 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.121569 13678 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:51.189803 13847 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:01:51.113878 13707 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.117497 13622 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.126785 13698 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.215317  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.217147  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2573) took 1624 ms. Trace:
I20260709 13:01:51.126184 13810 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.222836  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.223409  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.223681  8816 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
W20260709 13:01:51.224393  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 107148us
I20260709 13:01:51.226140  8824 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:51.227039 13698 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.227717 13769 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.228735 13630 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.228848 13852 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.229143 13698 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.229743  8893 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:51.222918  8821 rpcz_store.cc:276] 0709 13:01:49.593086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.594898 (+  1812us) service_pool.cc:224] Handling call
0709 13:01:49.595211 (+   313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:49.598812 (+  3601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.598840 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:49.610959 (+ 12119us) log.cc:844] Serialized 52 byte log entry
0709 13:01:51.032309 (+1421350us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b1800428d40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:01:51.073064 (+ 40755us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:51.073110 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.073180 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.192070 (+118890us) raft_consensus.cc:1685] finished
0709 13:01:51.192100 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.217080 (+ 24980us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3280,"raft.queue_time_us":3437,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":14481,"spinlock_wait_cycles":15618432,"thread_start_us":40573,"threads_started":1,"wal-append.queue_time_us":1538987}
I20260709 13:01:51.223021  8818 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
W20260709 13:01:51.223807  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 160646us
I20260709 13:01:51.115958 13843 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.236281 13417 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:51.236470 13833 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:51.236728 13723 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:51.238222 13848 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:01:51.215258 13428 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.229454 13852 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:51.240492 13724 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.650s	user 0.000s	sys 0.011s
I20260709 13:01:51.228735 13849 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:51.240502 13706 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.260669  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:51.249815 13808 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.005s
I20260709 13:01:51.256937  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.262832  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.263672  8891 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:51.245273 13417 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.271052 13852 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.230404 13853 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.265367  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.266781 13843 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.268972 13622 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.252872 13833 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:51.261229 13766 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.272670  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.258399 13783 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.274573  8891 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:51.273627  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.258565 13838 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.259591 13706 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.282186  8894 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.278394 13839 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:51.290434  8749 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:51.301950 13620 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.291532 13833 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.304847 13839 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.305824 13839 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.254325 13724 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:51.317559 13858 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:01:51.317785  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44312.4us
I20260709 13:01:51.317922 13862 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:01:51.318281 13842 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:51.321785 13824 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.278846 13696 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.328258 13652 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.331125  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:51.279338 13511 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.332643 13862 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:51.340019 13862 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:51.338970 13417 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.378631 13549 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.378664 13720 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:51.391506 13549 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.378815 13707 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.382212 13853 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.396502  8815 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:51.402447  8895 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:51.412262  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1943) took 1815 ms. Trace:
I20260709 13:01:51.416546  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:51.413789  8863 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.013s (SENT)
I20260709 13:01:51.418324  8822 rpcz_store.cc:276] 0709 13:01:49.596711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.596842 (+   131us) service_pool.cc:224] Handling call
0709 13:01:49.597328 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.254377 (+657049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.254406 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.254446 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:50.254472 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.412009 (+1157537us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540004ddd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:51.412057 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.412189 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510856960}
I20260709 13:01:51.402240  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.421941  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.422359  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.396498  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.423373  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.423468  8823 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:51.394827  8894 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:51.424000  8822 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:51.404533  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:51.425257  8749 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:51.418943  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:51.423187  8821 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:51.422748  8818 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:01:51.428038  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.428052  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.428202  8818 pending_rounds.cc:85] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 7
I20260709 13:01:51.429153  8821 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:51.429702  8818 pending_rounds.cc:107] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:01:51.430110  8818 raft_consensus.cc:2889] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.403592 13706 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.427399 13419 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:51.399369 13549 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:51.422762 13733 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.539s	user 0.015s	sys 0.000s
I20260709 13:01:51.405874 13557 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:51.431380 13419 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.397698 13720 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.432013 13733 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.431559  8816 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:51.435015  8823 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:51.435256  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.436304  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:51.411479 13677 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 1.033s	user 0.019s	sys 0.003s
I20260709 13:01:51.407860 13823 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:51.445506 13707 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:51.408027 13645 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.403836 13606 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.402988 13618 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.412241 13810 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:51.465775  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 201879us
I20260709 13:01:51.432122 13557 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:51.470337  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.471144  8891 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.445566 13678 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.468693  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.406757 13753 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.484505  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.485387  8894 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:51.475921  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.487450  8816 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:51.489758 13733 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.448925 13677 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.481796  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.449731 13707 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:51.504786  8891 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:51.508244  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.511469  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.512780  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.514528  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.514886  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.516083  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.516685  8891 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:51.517724  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.517828  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.518755  8818 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:51.519658 13618 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:51.520215 13618 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:51.521049 13618 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.521482 13812 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.521651  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.522069 13812 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.522992 13743 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.523394 13743 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.523458  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:51.523819 13743 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.524174 13812 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.524469  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33086 (request call id 4) took 1164 ms. Trace:
I20260709 13:01:51.525398 13664 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:51.525815 13664 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:51.526935 13810 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.526783 13664 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.527920 13414 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:51.528546 13414 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.516398  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.502023 13707 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:51.524852  9443 rpcz_store.cc:276] 0709 13:01:50.366311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.368815 (+  2504us) service_pool.cc:224] Handling call
0709 13:01:51.524097 (+1155282us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54006615a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f644c520 00007f90f64c9652 00007f90fb49760a 00007f90f4a1d7d4 00007f90f4a1d762 00007f90f4a1d691 00007f90f4a1d60c 00007f90f4a1c8c2 00007f90f480a48c 00007f90f48098c6
0709 13:01:51.524365 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637840128}
I20260709 13:01:51.530560  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.532141 13511 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:51.532545 13511 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.532902 13511 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.539058  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.527489 13810 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.544474 13414 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:51.544773  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.546092  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.546731  8891 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14. Re-sending same reply.
I20260709 13:01:51.503764 13419 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.524127  8821 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
W20260709 13:01:51.552062  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 838) took 1923 ms (client timeout 1617 ms). Trace:
W20260709 13:01:51.552593  8746 rpcz_store.cc:269] 0709 13:01:49.628275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.669961 (+ 41686us) service_pool.cc:224] Handling call
0709 13:01:51.551755 (+1881794us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540013c9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:51.551974 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2320608128}
I20260709 13:01:51.513455 13733 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:51.563867  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.563876  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.567387  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.516311 13606 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:51.569940  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.559610  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:51.551380 13853 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:51.573547  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.575460  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.556815 13737 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.579432  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.561167 13835 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.581090  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.551491 13549 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:51.561878 13664 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.584744  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.580314  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.582769 13852 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:01:51.566586 13733 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.570503 13415 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.588581  9442 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:51.569198 13606 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.572671 13838 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:51.575773 13850 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:51.589737 13535 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.589543 13739 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:51.594182 13737 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.587811 13677 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.601869  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.547430 13557 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.618181 13862 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.605043 13853 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:51.586772 13869 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:01:51.626873 13869 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:51.627283 13869 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:51.626595 13853 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.607244 13733 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.608769 13606 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:51.637112 13737 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.607748 13415 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.633994 13869 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:51.635430 13557 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:51.600474 13835 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.629429 13865 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:51.648505  8889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
W20260709 13:01:51.648991  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.649314  8889 raft_consensus.cc:606] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:01:51.646992 13865 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:51.651157 13865 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.653673  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 1916) took 4093 ms. Trace:
I20260709 13:01:51.599675 13549 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.653362 13261 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:51.674001 13261 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:51.674817 13826 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:51.675297 13826 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:51.674939 13261 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.654201  8747 rpcz_store.cc:276] 0709 13:01:47.570034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.821912 (+251878us) service_pool.cc:224] Handling call
0709 13:01:49.472309 (+1650397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:49.472752 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.472790 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:49.473453 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 13:01:49.565096 (+ 91643us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:49.685979 (+120883us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.116867 (+430888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.116982 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.319973 (+202991us) raft_consensus.cc:1685] finished
0709 13:01:50.320000 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.653452 (+1333452us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400021cd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:51.653593 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59605,"prepare.queue_time_us":125924,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":11159,"raft.queue_time_us":125848,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":19765,"spinlock_wait_cycles":3795221888,"thread_start_us":493566,"threads_started":2,"wal-append.queue_time_us":144295,"wal-append.run_cpu_time_us":4018,"wal-append.run_wall_time_us":1749458}
I20260709 13:01:51.640640 13415 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.642680 13862 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.679355  9447 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:51.625067 13839 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.652421 13737 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.684125  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 84620.4us
I20260709 13:01:51.686240  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.686796  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:51.689272  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 841) took 1255 ms (client timeout 1617 ms). Trace:
I20260709 13:01:51.692116  9446 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:51.692050  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:51.689884  8898 rpcz_store.cc:269] 0709 13:01:50.433526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.433670 (+   144us) service_pool.cc:224] Handling call
0709 13:01:51.686689 (+1253019us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:51.689185 (+  2496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077564288}
I20260709 13:01:51.681722 13869 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.699061  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1944) took 2101 ms. Trace:
I20260709 13:01:51.699468  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.701715  8817 rpcz_store.cc:276] 0709 13:01:49.599837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.625396 (+ 25559us) service_pool.cc:224] Handling call
0709 13:01:49.625789 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.732906 (+1107117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.698779 (+965873us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:51.698999 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542687104}
I20260709 13:01:51.702606  9442 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:01:51.703020  9442 pending_rounds.cc:85] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 10
I20260709 13:01:51.703219  9442 pending_rounds.cc:107] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:01:51.703436  9442 raft_consensus.cc:2889] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.705546  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.704850  9441 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 11 LEARNER]: Advancing to term 13
I20260709 13:01:51.679586 13743 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.676047 13826 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:51.714694  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 1944) took 2261 ms (client timeout 2013 ms). Trace:
W20260709 13:01:51.715137  8740 rpcz_store.cc:269] 0709 13:01:49.453727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.472011 (+ 18284us) service_pool.cc:224] Handling call
0709 13:01:49.559796 (+ 87785us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b5800010298. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:51.714498 (+2154702us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54001d9554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:51.714625 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717079168}
I20260709 13:01:51.716969  9440 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:51.718461  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.723814  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.724181  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:51.701910 13733 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.726137  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:01:51.726866  8898 raft_consensus.cc:606] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:51.727964 13733 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.728684 13707 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.730876 13707 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:51.737910 13871 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.148s	user 0.002s	sys 0.000s
I20260709 13:01:51.738058  8741 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:51.739588  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.740257  8898 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:51.745414  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.749624  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.750253  9440 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:51.750413 13835 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.751224  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:51.752120  8740 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:51.752526  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:51.751259  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.754359  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:51.691037 13737 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.755661  9442 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:51.751449 13826 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:51.757890 13743 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.758139  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.753382  9440 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:51.763814 13711 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:51.761185  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.768954  9440 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:51.769645  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.769815  8898 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:51.771023  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:51.764627 13851 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:51.777638 13419 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:51.779134 13862 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:51.771092  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.749012 13511 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.785018  9440 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
W20260709 13:01:51.786808  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.787082  9441 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 13 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:51.791855  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.793558  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.793867  9444 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:51.792050  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:51.794732  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:51.795771  9449 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:51.799165  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.800241  9441 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:51.801991  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.803114  9442 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:51.804625  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:51.786104 13743 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:51.809048  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:51.810828 13860 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:51.819897 13860 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.820312 13860 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.780838 13606 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.827123 13606 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.828281 13769 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:51.829110 13846 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.834s	user 0.002s	sys 0.002s
I20260709 13:01:51.828908 13769 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:51.829013 13860 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.830251 13769 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.831670 13544 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.832562 13860 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.780838 13865 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.837427  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.839337  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.808789 13733 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.841974  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.842589  9441 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:01:51.843919  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.847112  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.848001 13813 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.845204  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:51.850611  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.850689  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:51.851311  9447 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:51.851958  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.852166  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.852914  9447 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:51.854677  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.855862 13558 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.856078 13724 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.856266 13558 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.856681 13558 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.860826 13574 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.808450 13862 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:01:51.862167  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:51.840716 13707 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.874596 13839 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.863843 13733 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:51.874975 13707 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.875696 13733 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.877199 13707 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.878999 13558 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.879732  9441 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:51.880349 13720 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.845182 13261 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:51.891978  9446 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:51.870414  9448 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
W20260709 13:01:51.873806 13877 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:51.892004 13862 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:51.899984 13862 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:51.869172  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.900943 13862 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.904546 13862 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:51.907806  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.908416  8889 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.910006 13865 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.910463 13810 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:51.866541  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:51.910965  8891 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:51.911548  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45088.5us
I20260709 13:01:51.911923  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.912724  8713 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.913007  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.913223  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.913777  8897 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:51.914074 13810 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:51.914629  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.913584 13707 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.915199 13707 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.915552 13707 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.915812  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2279) took 1638 ms. Trace:
I20260709 13:01:51.839675 13743 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:51.918553 13743 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.882411  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:51.922120  8890 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:51.922669  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.916301  8896 rpcz_store.cc:276] 0709 13:01:50.277643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.277761 (+   118us) service_pool.cc:224] Handling call
0709 13:01:50.310464 (+ 32703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.362152 (+ 51688us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:50.362196 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.362791 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.849021 (+486230us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:50.849062 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.915431 (+1066369us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:51.915478 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.915526 (+    48us) raft_consensus.cc:1685] finished
0709 13:01:51.915542 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.915758 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2389,"raft.queue_time_us":193171,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12307,"spinlock_wait_cycles":112377088,"thread_start_us":303379,"threads_started":1,"wal-append.queue_time_us":869062}
I20260709 13:01:51.923414  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:51.923696  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 86407.9us
I20260709 13:01:51.924724  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:51.913646  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:51.926208  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.926939  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.881155 13558 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:51.883278 13429 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:51.955483 13429 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.957823 13429 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.887033 13511 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:51.888878 13506 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.915710  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.946695  9440 pending_rounds.cc:85] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 9
I20260709 13:01:51.961310  9440 pending_rounds.cc:107] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:01:51.961560  9440 raft_consensus.cc:2889] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.882679  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:51.952327  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:51.953651  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.957515 13733 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:51.886699  8898 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:51.979368  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96760.6us
I20260709 13:01:51.961666  8819 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:51.964417  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:51.934999 13765 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:51.940852 13810 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:51.953902 13826 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:51.968263  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:51.948436  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:51.993866  8890 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:51.962337  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:51.958540 13574 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:51.954463 13724 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:51.954494 13618 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.987936  8897 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:51.960119 13869 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:01:51.989045 13852 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:01:51.996073  8740 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:51.998101  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:51.992902 13621 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:51.995249  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:51.962164 13506 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:51.990168 13810 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.002902  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.003809  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:51.991322  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:51.999409 13865 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.023213 13865 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.024585 13506 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.020630 13810 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.028662 13813 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:51.993335 13429 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:01:52.030453 13709 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.030887 13709 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.031229 13709 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.034437  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:52.029186 13813 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:52.035629  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.036132 13813 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.036530 13678 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:52.036940 13678 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.002902 13724 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.008785  8819 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:51.987319 13598 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.966682 13557 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:01:52.009739 13852 consensus_queue.cc:1243] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:51.993351 13826 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.018852  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.007180  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.084822  8897 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:52.022200  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.036844  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:52.020844  8895 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:01:52.001056 13833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.006s	sys 0.012s
I20260709 13:01:52.046948  8818 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:52.063524  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.092391  9449 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:51.958920 13606 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:01:52.009706 13804 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.071178  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67472.3us
I20260709 13:01:52.081501  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.092573  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.008194 13618 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.064324 13621 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:52.175686  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68029.7us
I20260709 13:01:52.045957 13678 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:01:52.178864  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.179028  8896 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:52.174998  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.108815  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.045532 13724 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.045621 13429 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.090598 13825 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:01:52.101430 13833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.006s	sys 0.012s
W20260709 13:01:52.186123  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 188124us
I20260709 13:01:52.185791  9442 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:52.182090  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.128582  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:52.103410  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66656.9us
W20260709 13:01:52.102838  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80747.9us
I20260709 13:01:52.199395  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.101866 13506 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.101307 13574 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.081316 13823 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.203233  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.178409 13835 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.213792  9442 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:52.045805 13865 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.183704 13805 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.224624  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 178319us
I20260709 13:01:52.225404  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.226233  8896 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:52.229260  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.196882 13706 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:52.206761  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.184748 13838 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.209628  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.185062 13621 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:52.071750 13598 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.091209 13826 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.193121 13804 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.232411  8896 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:52.249996  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.248217  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2280) took 1965 ms. Trace:
I20260709 13:01:52.212878 13557 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.202553 13837 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:52.239538  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.240826  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.245324  8816 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 14. Ops: []
I20260709 13:01:52.246707  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.214612 13558 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.252449  8822 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.216216 13835 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.250938  8892 rpcz_store.cc:276] 0709 13:01:50.283153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.283381 (+   228us) service_pool.cc:224] Handling call
0709 13:01:50.294202 (+ 10821us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:50.295102 (+   900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.295142 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:50.346699 (+ 51557us) log.cc:844] Serialized 52 byte log entry
0709 13:01:50.676692 (+329993us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:52.034463 (+1357771us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800408fa0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf
0709 13:01:52.034589 (+   126us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.247689 (+213100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.247800 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.247859 (+    59us) raft_consensus.cc:1685] finished
0709 13:01:52.247958 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.248129 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3537,"prepare.queue_time_us":203312,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":32,"raft.queue_time_us":55530,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":6499,"spinlock_wait_cycles":2798464,"thread_start_us":669354,"threads_started":3,"wal-append.queue_time_us":487724,"wal-append.run_cpu_time_us":2309,"wal-append.run_wall_time_us":1033300}
I20260709 13:01:52.264622  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.265604  8821 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:52.253562  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.253736  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.257113  8896 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:52.253082  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.228354 13853 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.198745 13429 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.271312  8822 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:52.250022 13839 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:52.237473 13765 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.274730  8889 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:52.237905 13838 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:52.209983 13506 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.245462 13621 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.281059  8897 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:52.285392  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.270924 13835 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.230171 13823 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.286029  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.238077 13805 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:52.280278 13880 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:01:52.281728 13853 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.305987 13853 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.284514 13891 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.307031 13891 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.307397 13891 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:52.308336 13887 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:01:52.309423 13862 raft_consensus.cc:993] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:01:52.310067 13862 raft_consensus.cc:1081] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:01:52.310720 13598 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.311945  8816 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:52.248278 13706 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.313862  8892 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:52.281533 13733 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.289546 13838 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.282449 13837 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:52.319263  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.267226 13557 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.320497  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:52.266961 13558 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.327879  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.328940 13598 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.330169  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.331717  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.296129 13839 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.338781  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.295699 13765 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.327056 13869 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:52.343163  9442 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:52.348639 13415 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.349460 13415 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:52.350172 13415 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:52.352061 13810 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.352873 13558 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.380316 13707 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.382161  8892 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:01:52.388924  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 189627us
I20260709 13:01:52.344679 13812 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.390108  8892 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:52.346676 13869 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:52.394503  8747 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:52.396507  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.398224  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.403558  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.404232  8740 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:52.406214  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:52.406713  8741 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:52.414099  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82508.2us
I20260709 13:01:52.414927  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:52.390838 13506 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.398764 13869 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.423763  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.419409  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.426494  8822 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:52.405515 13812 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.416086 13805 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.425453 13618 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.437803 13415 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:52.447722 13898 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:52.448494 13549 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:52.449276 13895 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:01:52.449901  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:52.457711 13429 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:52.468122 13261 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.003s
I20260709 13:01:52.442034 13905 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:52.470113 13557 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.474406 13891 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.474877 13839 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.476393 13621 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.478008 13810 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.481570  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.482045  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2288) took 1413 ms. Trace:
I20260709 13:01:52.446949 13812 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.468220 13724 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.409127 13837 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:52.487092  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 79693.1us
I20260709 13:01:52.437824 13506 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.524674 13506 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:52.469758 13894 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:52.511535  9445 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:01:52.499173 13890 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:01:52.516520  8893 rpcz_store.cc:276] 0709 13:01:51.068507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.068980 (+   473us) service_pool.cc:224] Handling call
0709 13:01:51.079318 (+ 10338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.230088 (+150770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.481794 (+1251706us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400247a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:52.481985 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332134656}
I20260709 13:01:52.504343 13415 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:52.529677 13415 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.530287 13707 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.532212 13753 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.534029 13664 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:52.534559 13664 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:52.535436 13664 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.525101 13506 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.537353  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.536772 13678 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.538144  9447 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:52.539017  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.539841 13678 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.540139  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.543200 13720 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.543581  9449 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:52.547962  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.548610  8817 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:52.528226 13706 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.523938  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.553824  8892 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:52.511973 13906 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:01:52.555218  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.555923  8740 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:52.556931  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.557924  8822 raft_consensus.cc:2368] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15.
W20260709 13:01:52.559882 13813 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:01:52.560043 13621 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.565447 13765 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:52.568125 13883 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.292s	user 0.000s	sys 0.000s
I20260709 13:01:52.552620  9449 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:52.562325  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.562654 13899 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:52.550861 13506 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.582428  8817 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:52.504678 13598 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:52.536836 13664 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:52.568425 13835 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.560354 13813 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.526450  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:52.588836  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.590298  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.591377  8898 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:52.594465 13707 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:52.597635 13707 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.600458 13707 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.600999 13429 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.601614 13429 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.603114 13835 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.603832 13429 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.604466 13906 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:52.590637 13709 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.002s
I20260709 13:01:52.589231 13621 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:52.589452 13558 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.593024 13810 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.610096  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.611089  8898 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:52.512216 13905 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.612517  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.605939 13898 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.605386 13720 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.607746 13765 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.608572 13621 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:52.620395  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119060us
I20260709 13:01:52.608932 13558 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.605105 13906 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.627628 13826 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.628854  9443 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:52.630430 13737 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.621551 13733 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:52.609081 13709 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.626709 13558 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.647543  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.611706 13810 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.649135  8740 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:52.650949  8891 raft_consensus.cc:2368] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:52.625751 13765 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.622642 13720 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.620839 13898 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.647738 13912 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:52.664851  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.665910  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.655375 13429 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.671911  8744 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:52.651883 13733 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:52.673170  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.674528  8748 raft_consensus.cc:2368] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:52.675482  8744 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:52.675776  8749 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:52.676775  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.672683 13912 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.679179 13506 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.680305 13804 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.685813 13429 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.688052  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.695279 13765 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.699076 13905 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.714543  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 986) took 1014 ms. Trace:
I20260709 13:01:52.675086  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.715996  8742 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:52.714996  9446 rpcz_store.cc:276] 0709 13:01:51.700566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.700753 (+   187us) service_pool.cc:224] Handling call
0709 13:01:52.714362 (+1013609us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:52.714482 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427598080}
I20260709 13:01:52.721655  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.727612  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.730480  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.731077  9446 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:52.728905  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.732215  8817 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.732417  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:52.735535  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.737003 13813 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.738669 13558 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.739470  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.740178  9447 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:52.736222 13891 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.735541 13812 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.677434 13621 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.743350  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.743769  8817 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:52.746663  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.700711 13838 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.731907  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.718133 13826 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.767557  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.768312  8742 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:52.755424  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:52.769256  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.770046  8744 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:52.753418  9443 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:01:52.773689 13490 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:52.741254 13558 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.774699 13490 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:52.776114  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 863) took 1029 ms. Trace:
I20260709 13:01:52.776562  9440 rpcz_store.cc:276] 0709 13:01:51.746381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.807207 (+ 60826us) service_pool.cc:224] Handling call
0709 13:01:51.946505 (+139298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:51.993449 (+ 46944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.993498 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:52.003048 (+  9550us) log.cc:844] Serialized 52 byte log entry
0709 13:01:52.312347 (+309299us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:52.415885 (+103538us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.416380 (+   495us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.421104 (+  4724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.421208 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.464668 (+ 43460us) raft_consensus.cc:1685] finished
0709 13:01:52.464689 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.556462 (+ 91773us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18000e78e0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:01:52.775921 (+219459us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400663ad0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:52.776055 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34806,"prepare.queue_time_us":376132,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1078,"raft.queue_time_us":40781,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":16002,"spinlock_wait_cycles":330045184,"thread_start_us":319205,"threads_started":2,"wal-append.queue_time_us":385168}
I20260709 13:01:52.780011  9440 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:52.742082 13905 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.802855  9448 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:52.744781 13621 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.729168 13869 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.752852 13506 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.793061 13707 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:01:52.796267  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.786764  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.800745 13574 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.796646 13917 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:01:52.798907  9443 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:52.792776  9440 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:01:52.847347  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.847910  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:52.848765  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.797986 13906 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.799515  9442 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:52.782253  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.804316  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.795184 13368 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:52.806826  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.758906 13810 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.748178 13744 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:52.811067  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.866875  8745 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:52.818082  8891 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:52.854483  8895 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:52.868675  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.869419  8891 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:52.820281  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.870286  8892 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:52.820987  8889 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:52.821084  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.836393  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.871232  8893 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:52.805365 13905 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:52.845510  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44685.7us
I20260709 13:01:52.850147  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.850687 13765 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:52.849740  8742 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:52.793954 13898 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.874662  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.878736  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.851970  8740 raft_consensus.cc:2368] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:52.879324  8745 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:52.853235  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.882015  9440 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:52.882869  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.810921 13506 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.872498  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:52.873467  8896 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:52.854007  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.798486 13490 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:01:52.880795  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:52.852561  8744 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:52.845371 13574 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.852260 13368 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:52.896157 13368 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.896768 13490 consensus_queue.cc:1243] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:52.897271 13368 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:52.897424 13490 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:52.897899 13490 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:52.883920  8742 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:52.900386  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.901098  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:52.901537  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.901795  8740 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 10.
I20260709 13:01:52.901924  8742 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:52.902993  8740 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.884480  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:52.905790  9442 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:52.906304 13838 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:52.885655  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.908872  8894 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:52.846072 13606 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:01:52.909744 13869 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.910591  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.888176  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.912774  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:52.894673  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:52.913635  8744 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.898893 13490 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:52.911466  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:52.915822  8745 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:52.892478  9440 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:52.922251  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 994) took 1131 ms. Trace:
I20260709 13:01:52.781210 13733 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:52.929370  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.879918 13905 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:52.924970  8892 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:52.938613  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.939040  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:52.880115 13810 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:52.881700 13535 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:52.926226  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.923858  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.930630  8894 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:52.922634  9444 rpcz_store.cc:276] 0709 13:01:51.790925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.544994 (+754069us) service_pool.cc:224] Handling call
0709 13:01:52.778886 (+233892us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b5800164d98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:01:52.779085 (+   199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.779478 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.779518 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.780004 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.859356 (+ 79352us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:52.859492 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.859820 (+   328us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.910529 (+ 50709us) raft_consensus.cc:1685] finished
0709 13:01:52.910548 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.922196 (+ 11648us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15118,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":58116352,"thread_start_us":73426,"threads_started":1,"wal-append.queue_time_us":102085}
I20260709 13:01:52.931143  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.946347  8745 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:52.934348  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:52.852907 13511 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:01:52.853827 13813 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:01:52.894893 13506 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.941695  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:52.963268  8819 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:52.942638  8740 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:52.893904 13765 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:52.945222  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.876528 13783 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:01:52.947944  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.948014  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:52.976398  8898 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:52.948566  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:52.910935 13646 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:52.944975  8891 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:52.977531  8895 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.950328 13490 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:52.959627  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.960134  8744 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:52.882733 13744 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:01:52.960971  9442 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 16. Ops: []
I20260709 13:01:52.921739 13869 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.929718 13804 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:52.967163  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.998307  8740 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:52.991865  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:52.945788 13810 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:53.004664 13810 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:53.006558  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2508) took 1040 ms. Trace:
I20260709 13:01:53.006966  8897 rpcz_store.cc:276] 0709 13:01:51.966189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.231356 (+265167us) service_pool.cc:224] Handling call
0709 13:01:52.280805 (+ 49449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.297973 (+ 17168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.006350 (+708377us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b54000483d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.006496 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113066240}
I20260709 13:01:52.981746 13805 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:01:53.012109 13912 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.012465 13720 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.012784 13912 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.015743 13724 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:52.989910 13646 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.017726 13646 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.020283  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.021123  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 989) took 1303 ms. Trace:
I20260709 13:01:53.021848  9445 rpcz_store.cc:276] 0709 13:01:51.718017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.718152 (+   135us) service_pool.cc:224] Handling call
0709 13:01:53.020950 (+1302798us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54006dd654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.021065 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822149504}
I20260709 13:01:53.023064  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1965) took 1575 ms. Trace:
I20260709 13:01:53.023458  8815 rpcz_store.cc:276] 0709 13:01:51.448141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.448402 (+   261us) service_pool.cc:224] Handling call
0709 13:01:51.448805 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.451151 (+  2346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.451196 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.451767 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.753196 (+1301429us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b1800441580. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:01:52.936006 (+182810us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:52.936152 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.936250 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.022766 (+ 86516us) raft_consensus.cc:1685] finished
0709 13:01:53.022787 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.023005 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5436,"spinlock_wait_cycles":14188800,"thread_start_us":182564,"threads_started":1,"wal-append.queue_time_us":1566027}
I20260709 13:01:53.025918 13783 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:53.026508 13783 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.028565 13783 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.029883  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.031224  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 995) took 1240 ms. Trace:
I20260709 13:01:53.032066  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 874) took 1049 ms. Trace:
I20260709 13:01:53.031587  9446 rpcz_store.cc:276] 0709 13:01:51.791148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.735116 (+943968us) service_pool.cc:224] Handling call
0709 13:01:52.735480 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.735844 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.736139 (+   295us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.736628 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.873927 (+137299us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:52.873969 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.874033 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.031001 (+156968us) raft_consensus.cc:1685] finished
0709 13:01:53.031021 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.031166 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5625,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":80,"thread_start_us":136874,"threads_started":1,"wal-append.queue_time_us":262234}
I20260709 13:01:53.033521  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.035166  9445 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:01:53.035631  8895 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:53.002534  9440 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:52.948928 13535 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.039701  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.003204 13869 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.003764 13804 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.032598  9449 rpcz_store.cc:276] 0709 13:01:51.982539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.174689 (+192150us) service_pool.cc:224] Handling call
0709 13:01:53.031907 (+857218us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54006720d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.032004 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824167680}
I20260709 13:01:53.029074 13783 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:53.013146 13912 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.044409  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.045320  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.046196  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 856) took 1123 ms. Trace:
I20260709 13:01:53.041592 13869 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.041128 13535 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:53.047345 13574 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.048311 13724 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:53.048517 13765 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.050129 13618 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:53.041674 13804 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.051620  9415 leader_election.cc:400] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15.
I20260709 13:01:53.064388  9415 leader_election.cc:403] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:53.052932  8741 rpcz_store.cc:276] 0709 13:01:51.922959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.923102 (+   143us) service_pool.cc:224] Handling call
0709 13:01:53.045969 (+1122867us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400161054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.046129 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109587840}
I20260709 13:01:53.046219 13783 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:53.065833  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.066522  8741 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:53.066955  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.044448  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.053061  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.058163  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.065972  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:53.072101  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:53.057103  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.076889  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.078276  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.079793  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.082999  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.083000  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.084296  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.086175  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:53.051347 13895 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:53.053391 13912 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.088296  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.103230  8743 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:53.104203  8743 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.052022 13838 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:53.096609  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.064587 13724 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:53.098007  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:53.105482  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.063443 13618 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:53.077167 13415 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:53.108243  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.077080 13558 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:53.069437 13549 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15.
I20260709 13:01:53.114141  8740 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:53.063186  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 198360us
I20260709 13:01:53.097095 13826 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:53.114732  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.053383 13664 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 13:01:53.108119  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 311986us
I20260709 13:01:53.122927  8712 leader_election.cc:400] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:53.123733  8712 leader_election.cc:403] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:01:53.118717  8740 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:53.104060 13598 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:53.105165 13835 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.132789  8747 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:01:53.130995  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 202706us
I20260709 13:01:53.131913  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.126320  8743 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:53.106395 13838 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.131326  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.114526  8741 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:53.149322  8744 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:01:53.135703  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.173477  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.173622  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.174857  8740 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:53.175081  8743 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:53.175276  8740 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:53.169850 13919 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:53.122488 13558 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:53.188696 13812 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:53.189137 13812 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:53.184937 13677 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:53.190021 13415 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:53.190575 13415 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:53.190076 13812 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.191435 13812 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.191355 13415 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.192785 13415 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:53.134156 13926 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:53.170593 13724 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:53.130204 13598 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:53.114702 13506 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.183115  8740 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.176764 13618 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.193250 13646 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.194847 13724 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:53.205440 13724 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:53.205483 13804 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.206525 13724 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.207219  8749 raft_consensus.cc:2368] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:53.207731 13724 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.207798 13804 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.170392 13606 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:01:53.173298 13838 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.197890 13506 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.195930 13905 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.198028  8740 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.211773 13905 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.213949 13927 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:53.215276 13927 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:53.215705 13927 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.216902  8740 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:53.198282 13574 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.217536  8742 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:01:53.218605 13574 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:53.218977 13574 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.219308 13574 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.198580 13618 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.221786  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.222771 13724 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:53.223603  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.224720  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.198721  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.225872  8747 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:53.226006  8743 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:53.196187 13598 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.226977  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.227451  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:53.227916 13696 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:53.228317 13696 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:53.229009 13720 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.229512 13720 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.197855 13558 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.229933 13720 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.171844 13783 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.231781 13783 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.132514 13835 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.232813 13835 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.234021  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.234131  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.234784  8749 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:53.202304 13646 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.200215 13810 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.177532 13917 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.177151 13707 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:53.238071  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.243772  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.244585  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.244933  8741 raft_consensus.cc:2368] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14.
I20260709 13:01:53.252498  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.254441  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.256451  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.257193  8817 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:53.257850  8819 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:01:53.243155  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.249545 13490 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:53.241724 13917 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:53.184515 13903 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:53.262094  8815 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:53.266957  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.274363 13709 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.275878 13823 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.275717 13838 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.239862 13810 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.281183  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.281966 13917 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.282302  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.283015  8740 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:53.282892  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.283715  8743 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:01:53.283717  8815 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:53.284677  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.285324  8741 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:53.285557  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.263646 13905 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:53.286175  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.287014  8740 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:53.291021  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.313000  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.313496  8819 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:53.313939  8822 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:53.292906  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.289179  8815 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:53.289610  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.314977  8823 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:53.296255 13927 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.321810  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.322551  8818 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.296952 13429 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.297830  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.276978 13912 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.287324 13926 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:53.229107 13696 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.308089  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 944) took 1467 ms. Trace:
I20260709 13:01:53.331441  9443 rpcz_store.cc:276] 0709 13:01:51.840447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.775859 (+935412us) service_pool.cc:224] Handling call
0709 13:01:52.798566 (+ 22707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.846409 (+ 47843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.307886 (+461477us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54001982d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.308032 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51406208}
I20260709 13:01:53.307945 13835 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.309159  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.317006  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.317595  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.318414  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.344100  8823 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:53.319113  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.344821  8820 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:53.346308  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.323712  9441 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:53.326133  8742 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:53.328305  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:53.329851  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.240355 13720 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.335716  8818 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
W20260709 13:01:53.336301  8746 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.029s	sys 0.006s
W20260709 13:01:53.357972  8746 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.029s	sys 0.006s
I20260709 13:01:53.336740  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.243057 13598 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.342772  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.309748 13838 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:53.238597 13535 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.360283  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:53.274217 13574 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.362388  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.366365  8747 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:53.331022 13621 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:53.382897  8744 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:53.385157  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.386973 13826 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:53.350219 13927 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.393615 13927 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:53.395880 13927 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.397496 13826 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.402801 13912 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:53.403695  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 429902us
I20260709 13:01:53.407166 13927 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.407677 13927 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.408973 13511 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:53.409376 13511 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:53.409649 13927 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.410198 13511 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:53.411244  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.411670 13511 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.330246 13823 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:01:53.414501 13535 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.389309 13838 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:53.417580  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.417640 13815 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:53.419965  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.422065  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:53.427090  8820 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:53.429919  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.435156  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.366801 13818 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.383342 13739 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:53.438757  8820 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:53.441856  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.439518  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:53.447533  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 877) took 1165 ms. Trace:
I20260709 13:01:53.448928  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.449249  9447 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:53.452730  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.448057  9448 rpcz_store.cc:276] 0709 13:01:52.282612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.419063 (+136451us) service_pool.cc:224] Handling call
0709 13:01:53.447262 (+1028199us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540050f854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.447458 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864411904}
I20260709 13:01:53.438201 13912 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:53.456334  8789 leader_election.cc:400] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:53.445364 13813 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.495639  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.460177  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.496649  8789 leader_election.cc:403] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:53.438741 13818 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:53.460649  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 325064us
I20260709 13:01:53.486591  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.465727  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.467818  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:53.529531  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.533584  8892 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.540392  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.534194  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:53.584297  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50193.5us
I20260709 13:01:53.468199  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.445191 13739 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:53.450433 13827 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.474154  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.536550  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.588047  8747 raft_consensus.cc:2368] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:53.539119  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.534667  8891 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:53.588467  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.589099  8743 raft_consensus.cc:2368] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:53.589399  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.458508  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.541174  8742 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:53.541832  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.455011 13925 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:53.542546  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:53.542716  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84946.1us
I20260709 13:01:53.555581  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:53.446843 13835 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.597352  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.598925  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.594592  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.600136  8743 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:53.600827  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.594941  8889 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.595270  8748 raft_consensus.cc:2368] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
W20260709 13:01:53.533988  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77697.2us
I20260709 13:01:53.451007 13618 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:53.609380  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.444137 13744 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:53.507318 13912 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.495463 13927 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:53.579504  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.535063 13813 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:53.631064  8895 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.507581 13838 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:53.444185 13929 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:01:53.611241  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.566299  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.542012 13765 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16.
I20260709 13:01:53.572587 13818 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.591687 13930 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:53.499668  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.590539  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.503167  8862 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.591167  8891 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:53.605376  9443 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:53.608189  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:53.609916  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:53.444540 13869 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.584950 13933 raft_consensus.cc:2764] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:01:53.620898  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.634347  8889 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:53.595749 13941 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.633560  9445 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.638302 13753 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:53.644675  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:53.677975  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:53.635241 13912 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.595734 13837 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:53.683320  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:53.635079 13805 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:01:53.595932 13606 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:01:53.643565  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.597117 13810 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:01:53.679957  8896 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:53.639078 13838 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.658560 13930 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:53.612178 13925 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:53.684181  9446 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:53.636353 13368 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:53.710325  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:53.677364 13598 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.681128 13804 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:53.713722  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:53.715651  9443 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.692637 13574 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.713028 13804 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.679880 13941 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.597859 13827 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.596926 13733 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:01:53.655742 13720 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.673666 13739 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.726171 13804 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:53.726595 13925 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.727438 13838 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:53.728386 13942 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:01:53.726294 13936 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:01:53.730830 13827 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.734608 13941 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:53.737912 13813 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:53.737577 13930 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.740410 13912 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.595294  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:53.773972  8742 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
W20260709 13:01:53.784363  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 218091us
W20260709 13:01:53.585870  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 117816us
I20260709 13:01:53.799840  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.628888  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:53.673588  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 170545us
I20260709 13:01:53.803615  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:53.804062  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 175157us
I20260709 13:01:53.639142  8821 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:01:53.805418  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.592133  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.800952  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:53.812323  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 134298us
I20260709 13:01:53.807078  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.819919  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.805176  8815 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:53.809820  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:53.833467  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.834435  8820 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.834760  8815 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:53.807036  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.780980  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:53.807852  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:53.838348  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57767.7us
I20260709 13:01:53.838765  8821 raft_consensus.cc:2368] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
W20260709 13:01:53.839730  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 247642us
I20260709 13:01:53.840996  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.841382  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.834451  8819 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:53.843394  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:53.817099  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.814348  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.812117  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:53.816221  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:53.853899  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44074.6us
I20260709 13:01:53.854215  8824 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.854008  8864 leader_election.cc:400] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:53.855185  8864 leader_election.cc:403] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:53.687021  8895 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:53.863844  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 951) took 1299 ms. Trace:
I20260709 13:01:53.865732 13696 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.866575  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 940) took 2049 ms. Trace:
I20260709 13:01:53.867067  9447 rpcz_store.cc:276] 0709 13:01:51.816847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.740836 (+923989us) service_pool.cc:224] Handling call
0709 13:01:53.448986 (+708150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.449699 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.866048 (+416349us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540050f850. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.866312 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628096256}
I20260709 13:01:53.870422 13574 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:53.871536 13574 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:53.690074  9442 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.873673 13368 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.874526 13368 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.700287  9445 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:53.635613 13618 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:53.822986  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.856838  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:53.864486  9441 rpcz_store.cc:276] 0709 13:01:52.564525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.687714 (+123189us) service_pool.cc:224] Handling call
0709 13:01:53.863262 (+1175548us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:53.863527 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323911936}
W20260709 13:01:53.919824  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70912.5us
I20260709 13:01:53.880420 13506 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.924573  8893 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:01:53.925125 13578 raft_consensus.cc:2764] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:01:53.877844 13490 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:53.939350  8816 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:53.939112 13753 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:53.940254 13765 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.877236 13368 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:53.880009 13707 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:53.943166 13927 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:53.944308 13933 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:53.944738 13869 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.945122 13939 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:53.946615 13711 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:53.947348  8815 raft_consensus.cc:1217] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 8.7->[10.8-16.12]   Dedup: 13.10->[15.11-16.12]
I20260709 13:01:53.948184  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:53.948767 13827 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.950016 13837 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:53.957826  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75062.9us
I20260709 13:01:53.922520  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:53.959100 13905 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:53.959789 13574 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:53.960122 13906 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:01:53.964529 13898 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:53.983851 13765 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.965690 13805 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.871518  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:53.957909  9446 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:53.970363  9449 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:53.974790 13506 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.975490 13709 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.921967  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65234.7us
W20260709 13:01:53.967010 13943 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.290s	user 0.002s	sys 0.000s
I20260709 13:01:53.970044 13912 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:53.984308 13927 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.006402  8824 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:01:53.998139 13711 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:53.997119  8815 pending_rounds.cc:85] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:01:53.998366 13827 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.007056  9442 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:54.000190 13905 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:53.979022 13490 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:54.028273  8818 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:54.012799 13744 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:54.012645 13645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.023s
I20260709 13:01:54.008073 13898 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.022084 13506 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.014245 13765 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.030982  8815 pending_rounds.cc:107] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:01:54.040550  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:54.030792 13927 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.034119 13490 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:54.037812 13645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.004s	sys 0.023s
I20260709 13:01:54.032150 13711 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.038223  8820 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:54.050026  8815 raft_consensus.cc:2889] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.033831 13827 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.061851  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:01:54.078505  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 130547us
I20260709 13:01:54.079790  8863 leader_election.cc:400] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:54.080277  8863 leader_election.cc:403] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.081597 13941 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:54.082353 13851 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.083055 13851 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:54.083715 13851 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:54.084936 13959 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:01:54.088416  8892 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.548s	user 0.018s	sys 0.002s
I20260709 13:01:54.089828  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:54.081504  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.091341  9440 raft_consensus.cc:2368] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:54.081455  9443 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:54.089217  8892 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:54.054334 13805 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.095289 13753 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:54.096781 13805 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:54.098351  8895 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.100544  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:54.106457  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 249938us
I20260709 13:01:54.106860 13805 raft_consensus.cc:2764] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:01:54.108692  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.108853  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:54.113070 13805 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.057456 13958 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:01:54.114764 13905 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.115563  8822 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:54.116237  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67133.6us
I20260709 13:01:54.118333  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:54.118791  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:54.052635 13733 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:54.128352  8821 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:54.118567 13804 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.130550  8822 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:54.131539 13804 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.132148  8822 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:54.120349 13898 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.133003 13898 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.122421 13618 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.081326 13429 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.059679 13722 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:54.119627  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 248540us
I20260709 13:01:54.134466  8898 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:54.074642 13709 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.128626 13490 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.137589  8893 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:01:54.138957  9441 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:54.076768  8818 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 13:01:54.142638  8818 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:54.143064  8818 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.146775 13429 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:54.147344 13429 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:54.147760 13429 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.149829 13958 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.149978 13707 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.150851 13958 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.151079 13707 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.156185 13818 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.157411  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.175310  9441 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:54.159153  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.175881  9443 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:54.176728  9447 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:54.159888  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.177866  9440 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:54.178738 13926 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.179576 13926 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:54.180027 13818 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.160631  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.161669 13912 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.161962  9444 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:01:54.181928 13912 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.163154  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.183568  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.184177 13912 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.184515 13812 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:54.184896 13912 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:54.185523 13912 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.185921  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.163714 13906 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:01:54.185123 13812 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.187012 13912 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.187940 13812 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.189900 13709 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.164716  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.190630 13709 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:54.146808  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:54.190968 13709 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:54.191814  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45212.1us
I20260709 13:01:54.192083 13709 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.192662  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.194221 13929 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:54.195058 13929 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.165359 13733 raft_consensus.cc:2764] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:01:54.167721 13838 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:54.167853  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:54.196692  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 160959us
I20260709 13:01:54.197511 13804 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:54.168912  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.201138  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.168859  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50232.6us
I20260709 13:01:54.170332 13707 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.174031  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.190896  8818 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.190994  9448 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:54.166345 13958 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.204819 13711 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.206321 13838 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:54.210638  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.189800 13810 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:54.208596 13722 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:54.206944 13511 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.693s	user 0.016s	sys 0.000s
I20260709 13:01:54.225000 13722 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.109496 13925 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.107s
W20260709 13:01:54.278720  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 75623.7us
I20260709 13:01:54.208578 13733 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.281001  8818 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.284662  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:54.287425  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.212792 13851 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.209909 13818 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:01:54.223743 13810 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:54.304134  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 111540us
I20260709 13:01:54.214795 13827 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:54.320696  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 306877us
W20260709 13:01:54.322408  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 204317us
I20260709 13:01:54.318539  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.322821  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.299083 13709 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:54.335075  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43572us
I20260709 13:01:54.340214  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.330935  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.299755 13898 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:54.347929  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:54.349789  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119574us
I20260709 13:01:54.350219  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:54.351754 13535 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:54.352226 13813 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:54.352543 13706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.008s	sys 0.008s
W20260709 13:01:54.352763 13815 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.900s	user 0.004s	sys 0.012s
I20260709 13:01:54.372267  9444 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:54.304886 13926 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.377127  9443 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:01:54.375125  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:54.373198 13706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.008s	sys 0.008s
I20260709 13:01:54.395486 13927 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:54.403977  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57004.1us
I20260709 13:01:54.407379  9440 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:54.370888 13818 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.402861 13898 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.414453  8891 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.401485 13739 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:54.412607  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.415328  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.423776  9442 raft_consensus.cc:2368] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:01:54.412472 13419 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.427856 13419 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.428188  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.429261  8889 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:54.430636 13962 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:01:54.431685  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53260.5us
I20260709 13:01:54.433833 13917 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:54.435169  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 102664us
I20260709 13:01:54.436123  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.438875 13927 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.442287 13765 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.444766 13765 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.447070 13862 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:54.447499 13862 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.448439 13862 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.449450 13783 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:54.449689  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:54.451385  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.452387  8897 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:54.453269 13961 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:54.453893 13961 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.454905 13961 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.457432  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:54.459359  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2007) took 1041 ms. Trace:
I20260709 13:01:54.459878  8817 rpcz_store.cc:276] 0709 13:01:53.418199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.120455 (+702256us) service_pool.cc:224] Handling call
0709 13:01:54.120894 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.121365 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.121403 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.122091 (+   688us) log.cc:844] Serialized 52 byte log entry
0709 13:01:54.302961 (+180870us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:54.353105 (+ 50144us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.393701 (+ 40596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.393776 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.459137 (+ 65361us) raft_consensus.cc:1685] finished
0709 13:01:54.459158 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.459303 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40313,"prepare.queue_time_us":50222,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":50902,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":4917,"thread_start_us":180461,"threads_started":1,"wal-append.queue_time_us":283386}
I20260709 13:01:54.461369 13941 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.461995 13941 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.463987 13837 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:54.412680 13906 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:54.465557 13906 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.465808 13711 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:54.466375 13906 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:54.466785 13906 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.467159 13851 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:54.467969 13490 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:54.468614  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 118550us
I20260709 13:01:54.414309 13912 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:01:54.473413 13912 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:54.473959 13912 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:54.474365 13912 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.474470 13827 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:54.475383 13912 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:54.476492  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.476812 13912 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.477608  8896 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:54.477451 13810 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.480755  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:54.481545 13898 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.482571  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:54.487826  8898 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:01:54.489166  9441 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:54.489251  9440 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:54.489796  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.472512 13506 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.491158  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:54.491699  8894 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:54.491969  8894 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.492111 13506 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.493635  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.494834  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:54.495460  8895 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:54.495357  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 1991) took 1081 ms. Trace:
I20260709 13:01:54.496237  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:54.496862  8895 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.497032  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:54.495785  8815 rpcz_store.cc:276] 0709 13:01:53.413952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.836088 (+422136us) service_pool.cc:224] Handling call
0709 13:01:53.836950 (+   862us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:01:54.137078 (+300128us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:54.177967 (+ 40889us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.353159 (+175192us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.394150 (+ 40991us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.404124 (+  9974us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.416847 (+ 12723us) log.cc:844] Serialized 52 byte log entry
0709 13:01:54.417054 (+   207us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:54.417508 (+   454us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.425017 (+  7509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.425131 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.495167 (+ 70036us) raft_consensus.cc:1685] finished
0709 13:01:54.495185 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.495305 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114151,"prepare.queue_time_us":83612,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1437,"raft.queue_time_us":75509,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":27874,"spinlock_wait_cycles":15718912,"thread_start_us":122727,"threads_started":1,"wal-append.queue_time_us":170991}
I20260709 13:01:54.493722  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:54.492719  8894 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.508466  8823 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:54.512758  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:54.513185  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 174689us
I20260709 13:01:54.513581  8896 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.513864  8894 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.514173  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.515473  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.516763  8897 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:54.518123  8897 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.520583  9448 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
W20260709 13:01:54.520783  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.521423  9440 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.564630  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:54.580753  8894 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:54.585819  8897 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:01:54.587163  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.552220 13917 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.591771  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.596879 13927 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.598258  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 165816us
I20260709 13:01:54.554872 13925 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:54.562074 13765 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:54.524024 13818 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.603992  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.604692  8894 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:54.467727 13906 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:54.616464  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 190753us
I20260709 13:01:54.618024  8862 leader_election.cc:400] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:54.622213  8862 leader_election.cc:403] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.595055 13917 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:54.619792  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 104410us
I20260709 13:01:54.626170  8896 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:54.614609 13862 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:54.601459 13511 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.644716  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.581974 13739 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:54.648247  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 54392.5us
I20260709 13:01:54.648720  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:54.608163 13927 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.602381 13429 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.642884 13917 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.660972  8897 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:54.605417 13744 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:01:54.665323 13906 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:54.673730  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58099.3us
I20260709 13:01:54.674675  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:54.677086 13739 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:54.681922 13932 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.288s	user 0.002s	sys 0.001s
I20260709 13:01:54.682422 13818 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.683732  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2225) took 1733 ms. Trace:
I20260709 13:01:54.684486 13927 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.684146  8745 rpcz_store.cc:276] 0709 13:01:52.949790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.950112 (+   322us) service_pool.cc:224] Handling call
0709 13:01:52.950522 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.950914 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.950955 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.951512 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.283897 (+332385us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:53.283934 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.284027 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.560917 (+1276890us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:54.683215 (+122298us) raft_consensus.cc:1685] finished
0709 13:01:54.683237 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.683528 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63005,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":197,"spinlock_wait_cycles":88194816,"thread_start_us":331912,"threads_started":1,"wal-append.queue_time_us":441948}
I20260709 13:01:54.687000  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:54.688736  8862 leader_election.cc:400] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:01:54.689080  8862 leader_election.cc:403] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.689600  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.658969 13804 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:54.691185  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.686849 13646 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:54.695501  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:54.697983  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44456.6us
I20260709 13:01:54.696219  9444 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:54.665989 13898 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.701951  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.665735 13869 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:01:54.716378  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.698258 13646 consensus_queue.cc:1243] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:54.708012 13837 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:01:54.694154 13927 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:01:54.711216 13925 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:54.713320 13933 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:54.725124  8895 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:54.725797 13933 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:54.727133 13933 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.729382 13960 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.729861 13744 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.731457 13724 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:54.733215  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:54.737437 13978 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:54.737826  8894 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:54.737859 13978 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.738426 13978 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.738592 13535 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:54.741750  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:01:54.742224  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67679.3us
I20260709 13:01:54.743261  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.745543 13812 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.746889 13960 raft_consensus.cc:2764] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:01:54.717820 13912 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:54.749768 13912 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.750839 13912 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.743289  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:54.754279 13869 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:54.754864 13869 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.755939 13869 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.759541  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.760502  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.760962 13812 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:54.761631 13812 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.761821  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:54.762627 13744 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:54.762609 13812 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.763633  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:54.764250 13744 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:54.765182 13574 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:54.765512 13898 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:54.766095 13898 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.767088 13898 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.725029 13925 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.768664 13925 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.765930 13744 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:54.712319 13429 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.765137  8789 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.763036  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:54.774003  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:54.719293  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:54.775902  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 56694.4us
I20260709 13:01:54.778848 13939 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:54.780952  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:54.782537  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:54.754302  8863 leader_election.cc:400] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
I20260709 13:01:54.789537 13978 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.790125 13733 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:54.790549 13733 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.790877 13733 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:54.795280 13978 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.797657 13905 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.803570 13838 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:54.804147 13838 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:54.807523  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 1012) took 1210 ms. Trace:
I20260709 13:01:54.808045  9449 rpcz_store.cc:276] 0709 13:01:53.596932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.597049 (+   117us) service_pool.cc:224] Handling call
0709 13:01:54.807090 (+1210041us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:54.807434 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609022336}
I20260709 13:01:54.814100 13978 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.814494  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.814909  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.814770 13978 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.817404 13978 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.817816 13724 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:54.818553 13724 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.820562 13419 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.720347 13783 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:01:54.199657  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:54.821770  8822 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:54.858891  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 93894.9us
W20260709 13:01:54.859213  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96289us
W20260709 13:01:54.859189  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 78357.2us
I20260709 13:01:54.897087  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:54.870136  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 108869us
I20260709 13:01:54.897928  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.829867  9444 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:01:54.841122  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 95812.5us
I20260709 13:01:54.836385 13765 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:54.860024 13429 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:54.909142 13765 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:54.926981 13429 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:54.927620 13765 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.928049 13429 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.928995 13765 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.831367 13744 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.873939  8645 catalog_manager.cc:5697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 10, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:54.858734 13606 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.932986 13606 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.933662 13606 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:54.848013 13963 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.654s	user 0.004s	sys 0.000s
I20260709 13:01:54.913648  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:54.935997 13978 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.936467  9441 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:54.859411 13574 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:01:54.868146  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2506) took 2908 ms (client timeout 2141 ms). Trace:
W20260709 13:01:54.937932  8890 rpcz_store.cc:269] 0709 13:01:51.959658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.128299 (+168641us) service_pool.cc:224] Handling call
0709 13:01:54.867889 (+2739590us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400094554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:54.868035 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5178173184}
I20260709 13:01:54.866564 13368 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:54.915845 13981 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.853974  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2005) took 1436 ms (client timeout 1554 ms). Trace:
W20260709 13:01:54.940397  8823 rpcz_store.cc:269] 0709 13:01:53.417747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.799058 (+381311us) service_pool.cc:224] Handling call
0709 13:01:54.853744 (+1054686us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540004d8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:54.853881 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987747712}
W20260709 13:01:54.921698  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 167633us
I20260709 13:01:54.923156 13917 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:54.925508  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:54.872838 13838 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.926059 13939 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:54.835501 13815 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:54.936543 13978 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.825191 13839 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:54.939060 13368 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.821188 13724 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:54.965345  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:54.966185  8822 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.968514 13929 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:54.968969 13929 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:54.969362 13929 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.974352 13618 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.974561 13917 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.838737 13941 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:54.956069 13815 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.946240 13927 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:54.998947  8822 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:54.960355 13839 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:54.980254 13838 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:55.008311  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 279138us
I20260709 13:01:54.974959 13574 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:55.036249  9444 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:55.039178  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.042235  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:54.984714 13939 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:54.975003 13709 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:01:54.957948 13981 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:54.978296 13618 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.069931  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.073943  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.083662  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.041646 13744 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.089214  9441 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:55.095252  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.095860  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:55.097260  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.069808 13927 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:55.098623  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.101193  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.102411  8896 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:55.103657  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.036763 13851 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:01:55.100505  8893 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:55.073171 13906 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.110010  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.110795  8895 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:01:55.048774 13707 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.117944  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.118654  8898 raft_consensus.cc:2368] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:55.120821  8892 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:55.121827  8818 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:01:55.092000 13941 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.128762  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.129451  8896 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.053131 13958 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:01:55.134479  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.132690  8897 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:55.102200 13927 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:55.094847 13724 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.143116  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.146858  8898 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:55.149598  8890 raft_consensus.cc:2368] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:55.131709  8889 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:55.124136 13645 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.159544  8749 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:55.161692 13906 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.163956 13906 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.155989 13941 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.177500 13941 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.178165 13724 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.180326 13724 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.182723 13618 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.185483 13939 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.188987 13645 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.189487 13645 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:55.193883 13985 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:01:55.195950 13927 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.197271 13927 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.201584 13978 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.149748 13574 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.206163  8892 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:55.208391 13707 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.210561 13707 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.134467 13839 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.214630 13839 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.104224 13744 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.219087 13744 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.119889 13429 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:01:55.094729 13368 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.222541 13733 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.235383 13606 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:55.241169  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 96685.3us
I20260709 13:01:55.246508  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.247252  8740 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.247656  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.248435  8744 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:55.246686  8749 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:01:55.249418  8749 pending_rounds.cc:85] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:01:55.249786  8747 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:55.249699  8749 pending_rounds.cc:107] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:01:55.256114  8749 raft_consensus.cc:2889] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.150766 13706 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.265187  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.276360  9444 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:01:55.249617  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:55.281348  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 332463us
I20260709 13:01:55.298931  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:55.295497  8863 leader_election.cc:400] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14.
W20260709 13:01:55.310271  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.323621  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.326567  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:55.328524  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.329270  8822 raft_consensus.cc:2376] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13. Re-sending same reply.
W20260709 13:01:55.331748  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.250593 13925 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.285198 13706 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:55.350248  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.351363  8789 leader_election.cc:400] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14.
W20260709 13:01:55.346174  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.345564  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.350956  8817 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:55.367792  8821 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.346081  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:55.341848  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60067.6us
I20260709 13:01:55.349393  8741 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:01:55.285435 13419 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:01:55.369544  8741 pending_rounds.cc:85] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 11
I20260709 13:01:55.381122  8741 pending_rounds.cc:107] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:01:55.381481  8741 raft_consensus.cc:2889] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.394060  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:55.394491  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.397146  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.289743 13645 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:55.397974  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:55.398896 13905 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.399097  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.399612  8815 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [12.9-13.10]
I20260709 13:01:55.399814  8891 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:01:55.400189  8891 pending_rounds.cc:85] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 11
I20260709 13:01:55.341444 13606 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.400435  8891 pending_rounds.cc:107] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:01:55.401546  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:55.401846  8891 raft_consensus.cc:2889] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.340471 13998 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.342494 13707 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.341394 13429 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:55.349742 13937 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.401000  8821 raft_consensus.cc:2368] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:55.402639  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.409487  8863 leader_election.cc:400] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:55.409852  8863 leader_election.cc:403] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.340034 13724 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.409791  8892 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
W20260709 13:01:55.399554  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68509.3us
I20260709 13:01:55.414968  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.417687  8820 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:55.418628 13724 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.420771 13724 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.421602  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.422557 13998 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.422712 13937 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.424628 13998 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.424929 13937 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.425406 13707 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.425769  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:55.426442  8823 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.429780  8740 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:55.399533 13905 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:55.432533 13991 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:01:55.433964 13905 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.434710  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:55.437588  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.438194 13724 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.438341  8749 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:55.438820 13724 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.439378 13724 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.440385 13925 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:55.425446 13989 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:01:55.414100 13898 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:55.448024  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.453668  8821 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:01:55.414994 13733 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.449822 13646 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:01:55.471580  8820 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:55.447587 13939 raft_consensus.cc:2764] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:01:55.480603  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.480268  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.485360  8747 raft_consensus.cc:2410] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:01:55.488276  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62643.6us
I20260709 13:01:55.489143  8744 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 13:01:55.493599  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.494356  8742 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:55.502418 13707 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.504082 13645 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.505780  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:55.507162 13929 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.002s
I20260709 13:01:55.507565 13929 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.473961 13898 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:55.509863  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:55.510159  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.510030 13898 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.510759  8820 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.512831 13722 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:55.486452 13937 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.513311 13722 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:55.513443  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:55.513819 13937 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.514406 13722 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.515435 13927 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:55.515944 13937 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.516902 13368 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:55.514783  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:55.630198  8820 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:55.515477 13722 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.637979 13838 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.649665  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.649763 13838 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.650874  8744 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
W20260709 13:01:55.651863  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 137162us
I20260709 13:01:55.652429 13838 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.652959  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:55.657522 13927 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:55.658185 13927 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:55.658638 13927 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:55.638332  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 311989us
W20260709 13:01:55.626463 14004 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:01:55.640930  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 236391us
W20260709 13:01:55.642218 14002 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:01:55.642868  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 133113us
I20260709 13:01:55.645579  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.674062  8824 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:55.665658  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:55.632632 13869 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:55.637768 13937 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.637557 13733 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.663555 13929 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.698463  8712 leader_election.cc:400] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:55.698905  8712 leader_election.cc:403] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.690392  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.663797 13783 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:55.695056  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.697172  8863 leader_election.cc:400] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 10 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11.
I20260709 13:01:55.697558  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.704443  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:55.705868  8742 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:55.707890  8712 leader_election.cc:400] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:55.708568  8712 leader_election.cc:403] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:01:55.709697  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 123063us
I20260709 13:01:55.693507 13869 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:55.711036 13869 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.713773  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.693771 13724 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.715538 13997 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:55.716135 13724 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.716470  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.717592 14008 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:55.718349  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:55.697793 13937 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.720443 13937 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.721024 13810 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.721483 13917 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:55.722926 13917 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.723286 13917 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:55.723351  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.723788 13917 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.724146  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:01:55.725481 13707 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:55.691944 14001 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:01:55.721511  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.731925 13810 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11.
I20260709 13:01:55.732347  8892 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:55.732963  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.735433  8645 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:55.735545  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:55.740540  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43512.4us
I20260709 13:01:55.698480 13898 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:55.745498  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.746209  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.724969 13733 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.740134 13939 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.767838  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.757843  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.770970  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.771027 13927 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.790166 13939 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.791488 13927 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.792001 13917 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.792420 13939 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.793853 13724 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:55.793910  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.794929  8744 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:55.794616 13724 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.795782  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.796401  8744 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:55.796818 13724 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.772543 14008 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:55.764101  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.757499  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.767056 13706 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.761891  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.797276  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.800439  8744 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:55.716140 13997 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.802222 13981 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.802877 13981 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:55.804071 13997 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.734035 13838 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:55.816067  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.817904  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.774603  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.738015 13937 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:55.763500  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.822677  8747 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:55.762362  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.801308  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.761723  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.823843  8815 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:55.763522  8821 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:01:55.784196  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.780287  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:55.763201  8816 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:55.789222 13958 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.789810  8820 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.798543  8746 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:55.799005  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.772931  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.799664  8745 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.762786  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.832904  8824 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:55.830468  8742 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:55.835779  8748 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:55.848388  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.781126 13898 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:55.851562  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.856763  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 76613.3us
I20260709 13:01:55.829592 13937 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:55.863864  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.867100  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:55.869696  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 134257us
I20260709 13:01:55.859965 13898 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.860978 13862 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.833847 13838 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.875365  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.874169  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:55.880672  9444 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:55.865756 13706 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:01:55.876335  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 163836us
I20260709 13:01:55.884469  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:55.885826  9446 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.876461  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.887532  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.863794 13937 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:55.873899 13851 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.887414  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:55.890045  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:55.872035  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.890709  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:55.890707  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.890514 13706 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:55.863950 13958 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.878535  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.909499 13706 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:55.896260  8740 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.910826  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.903411  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.914108  8894 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 11.
I20260709 13:01:55.904057  9441 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:55.904951  8819 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:55.916340  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.917068  8819 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.907521  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:55.917981  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:55.920648 13939 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.921020 13939 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.921166 13961 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:55.921458 13939 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.904034 13917 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.916950  8823 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:55.904368  8748 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.923015  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:55.924223  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.924700  8749 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:55.924887  8819 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:55.897184 13851 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.926049  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:55.926604  9441 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:55.927064 13851 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.928031  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:55.928954 13765 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.929425 13765 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.910851 13958 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:55.929876 13765 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.931676 13838 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.932247 13838 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.932593 13838 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.932900 13838 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.884351 13927 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:01:55.934594  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:55.893899 14020 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:55.905205  8715 leader_election.cc:400] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12.
I20260709 13:01:55.935861 14020 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:55.936578 13917 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:55.936787 14020 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.938153  9446 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:55.938258  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:55.939623 14008 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:55.940120 14008 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:55.940495 14008 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:55.939577  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:01:55.940953 13733 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:55.941654 13958 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:55.942394 13958 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:55.882611 13862 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:55.943208 13958 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.944586 13862 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:55.948376 13765 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.952229  8743 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:55.952730  8740 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:55.953245  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:55.953687  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:55.954847  8816 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:55.978627 13929 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:01:55.956084  8744 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 11.
I20260709 13:01:55.956182  8820 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:55.960259  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:55.975366  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 240765us
I20260709 13:01:55.903626 13937 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:55.979823  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:55.982335  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.981614  8821 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:56.001320  8740 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:55.982963  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.981927  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:56.002600  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:55.983527  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.986907  8744 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:55.992794  8745 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:55.994005  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.005668  8746 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:55.998519  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.017969  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:56.009647  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:55.983412 13929 consensus_queue.cc:1243] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:55.993162 13709 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:01:55.996903  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110152us
I20260709 13:01:55.981865 13724 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:56.025100  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:55.985903 13783 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:56.021698  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:56.017040 13958 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:56.029211  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:56.020053  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 100206us
W20260709 13:01:56.032859  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.994086 13813 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:01:56.035053  8789 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:56.018330  9449 raft_consensus.cc:2368] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:01:56.042496  8898 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:56.044662  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.023206 13927 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.026443 14021 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:56.060137  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.061028  8816 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:56.046568  8895 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.057842  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.063347  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.066771  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.067147  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:55.982858 13815 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.069157  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.069726  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.045858 13733 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:56.067404  8821 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:56.070910  8890 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:56.045358 13839 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.078969  8897 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:55.989439 14008 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.084301  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:55.987417 13898 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:01:56.096304  8749 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.097574 13898 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.097527 14033 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:56.046419 14028 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:56.065358 13709 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.099130 14033 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:56.099247 13709 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.094568  9440 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:56.082512 13783 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.093861  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:56.101835  8746 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:01:56.102044 13783 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.102638 13783 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.102988 13783 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.103075  8746 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:56.103375 13783 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.104471  8746 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [13.6-14.7]
I20260709 13:01:56.047101 13724 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.105798  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:56.038538 13941 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:01:56.047632  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:56.095068 14008 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.099648 14033 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:56.107949  8742 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 9.
I20260709 13:01:56.105388 13815 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:56.087260  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:56.116623  9440 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:56.131036  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43911us
W20260709 13:01:56.120179  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72641.3us
W20260709 13:01:56.129894  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49843.5us
I20260709 13:01:56.117311  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.133970  8895 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:56.134155  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:56.131846  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.135354  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:56.137465  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 102544us
W20260709 13:01:56.137645  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 70589.2us
I20260709 13:01:56.139899  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.141536  8822 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:56.116083 13929 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.144470  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.115794 13912 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:56.133275 13898 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.158380  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:56.132558 13368 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:56.106877 13839 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.118882 13709 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.083163 14019 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:01:56.153565 14020 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.153465  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.153956  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:56.167014  8788 leader_election.cc:400] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:01:56.172906  8788 leader_election.cc:403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:56.173017  8892 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:56.173499  8892 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:56.170526 13765 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.168324  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:56.142696 13813 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.148102 13578 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:56.145197 14032 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:56.177690  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 148612us
I20260709 13:01:56.178830  8895 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:56.179034 14035 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:56.152532  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.154321 13733 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:56.167021 13368 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:56.174192  8892 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.190485  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:56.310808  9440 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.167141 14008 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.191048  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.215783  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:56.313577 14008 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:56.314164  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98519.2us
I20260709 13:01:56.315151  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:56.315992 13941 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:56.316308  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:56.316478 13941 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.317070 13941 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:56.234972 13724 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.240270 14040 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.231148  8897 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:01:56.221004  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68553.4us
I20260709 13:01:56.366700  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:56.367726  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.175382 13912 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.220264 14032 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:56.369331 14032 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.191573 14035 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.286216 13706 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:56.263775  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 876) took 1140 ms. Trace:
I20260709 13:01:56.179558 13813 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.381145 13706 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.167677 14031 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:01:56.381711 13706 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.383561  8745 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:56.383919 14031 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.384492 13839 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.384464 14031 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.220032 13937 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.385633 13937 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.386094 13937 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:56.239805 14026 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:01:56.232646 13933 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:56.216416  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.394496  8895 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:56.381224  8741 rpcz_store.cc:276] 0709 13:01:55.123161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.133656 (+ 10495us) service_pool.cc:224] Handling call
0709 13:01:55.164723 (+ 31067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.406373 (+241650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.193324 (+786951us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:56.263703 (+ 70379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167865600}
W20260709 13:01:56.249475 14027 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:01:56.277715  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.396715  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.172389 13898 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.381382 13813 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:56.403601  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.158811 13929 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.314313 14028 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:56.289686  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.325621  8892 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.339035  9440 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:56.316346 14008 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.374950 13733 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:56.375655 14030 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:01:56.256083 14033 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.379509 14035 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.379777 13939 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.522198  8741 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:01:56.391366 14036 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:56.391384 13997 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:56.214303 13368 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.402015  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:56.410790  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:56.547114  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 136451us
I20260709 13:01:56.406401 14028 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.517932 14048 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.393149 13722 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:01:56.544607  8748 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:01:56.416131  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 282085us
I20260709 13:01:56.425733 14020 raft_consensus.cc:2764] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:01:56.552373  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 336545us
I20260709 13:01:56.552819  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:56.523341 14027 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:01:56.556143  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 420883us
I20260709 13:01:56.514532 13937 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:56.559032  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.562671  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:56.560225 14048 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:56.563408  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 195772us
I20260709 13:01:56.540814 13709 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.543592 14036 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.553975 14028 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.578461  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:01:56.567720  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 413807us
I20260709 13:01:56.554366 14008 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:56.598562  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:56.603829  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:56.583160 13838 consensus_meta.cc:287] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 13:01:56.577543 14036 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.610956  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.612383  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:56.623857  8819 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.613773  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.618098  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:56.613875  8892 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:01:56.578536 13937 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:56.584658 14032 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.624641  8821 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:56.618111  9443 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:56.621533  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.584376 14008 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:56.605646 13927 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:56.628512  8820 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:56.630123  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.627497  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:56.644774  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:56.645220  8821 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15.
I20260709 13:01:56.625566  8892 pending_rounds.cc:85] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 10
I20260709 13:01:56.646081  8892 pending_rounds.cc:107] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:01:56.646420  8892 raft_consensus.cc:2889] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:56.579892 13709 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.651968  9448 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:01:56.654263  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50517us
I20260709 13:01:56.658428  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:56.641618 14008 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.664590 14049 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:56.669723  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:56.670763  9443 raft_consensus.cc:2368] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:56.672475  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:01:56.672915 14050 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:56.673141  9442 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.674413  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.678109 13898 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:56.648835 14025 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
I20260709 13:01:56.646306 14024 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:56.629891 14047 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.626495 14032 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:56.699328  8890 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.001s
I20260709 13:01:56.707053  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:01:56.708745 14019 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:56.709844 14041 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:01:56.709594 14019 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.708788  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:56.712370  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:56.712558  9445 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:56.713531  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:56.714291 14019 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.708910 14018 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:01:56.715278 13815 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.001s
I20260709 13:01:56.715858  9445 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:01:56.627323 13838 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.718492 14053 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:56.719626 14064 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.720336 13838 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.720157 14064 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.682772 13574 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.722303  9449 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:01:56.724063  9442 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:01:56.733435  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 331551us
W20260709 13:01:56.732313  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 151461us
W20260709 13:01:56.742684  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84389.5us
I20260709 13:01:56.697211 14024 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:56.722414 14032 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.681269 13706 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.754985 13941 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.734436  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:56.765048  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:56.780026  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.722114 14047 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.750501 14051 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.770813  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:56.762217 14021 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.001s
I20260709 13:01:56.736097 13574 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:56.795864  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 217512us
I20260709 13:01:56.769745 13941 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:56.809466 13783 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
W20260709 13:01:56.825317  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55019.1us
I20260709 13:01:56.771708 13939 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:56.837628  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.750828 13937 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:01:56.825472 14024 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.861526  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:56.850394 14065 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:56.768424 13898 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:56.854573  9446 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:56.859856  8819 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:01:56.870850  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:56.823108 14021 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:56.861588 13958 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.807s	user 0.011s	sys 0.000s
I20260709 13:01:56.823319 14063 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.875639  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:56.880959  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.881853  8890 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:56.863739 14065 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.883879  9441 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:56.865772 13898 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.873868 13997 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.888159  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:56.908152  8745 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.890499  8893 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:56.886694  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:56.889668  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.863312 13927 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:56.891188 13722 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.911489 13927 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.888352  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:56.824945 13783 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.915752 13783 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:56.872112 13939 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.890255 14052 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:01:56.912778 13722 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.878156 14024 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.876375 14048 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.919107  8889 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:56.877395 13958 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:56.892958 14046 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:01:56.889703 14061 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:56.899297 14065 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:56.884285 13827 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.892704 14067 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.983186 14028 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.888813 14058 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:56.890316 14064 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.943533 14049 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.904964 13898 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:56.947151  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2267) took 1164 ms. Trace:
I20260709 13:01:56.947824 14056 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:56.938722 13815 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:56.949229  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:56.985526  9415 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.550s (SENT) [suppressed 18 similar messages]
I20260709 13:01:56.955448 14035 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.934052 13939 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.956354 14073 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:01:56.926331 13941 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.957536 13706 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:56.958086 14069 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:56.923421 13997 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.927450 14031 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.886934 14066 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:56.888574 14045 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:56.891623 14047 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:56.892582  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.966984  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:56.971698  8890 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:56.973462  8745 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:57.029639  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.030436  8745 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:56.979909 13933 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.000s
I20260709 13:01:56.982754 13722 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:56.983330 14067 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.947137 14044 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.984652  8820 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:01:56.984302 14058 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.986984 14049 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:56.987366  8747 rpcz_store.cc:276] 0709 13:01:55.783076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.404044 (+620968us) service_pool.cc:224] Handling call
0709 13:01:56.450069 (+ 46025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.509197 (+ 59128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.509229 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.546247 (+ 37018us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.854884 (+308637us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:56.855137 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.856142 (+  1005us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.946899 (+ 90757us) raft_consensus.cc:1685] finished
0709 13:01:56.946922 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.947062 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62188,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":24007,"thread_start_us":273698,"threads_started":1,"wal-append.queue_time_us":372890}
I20260709 13:01:56.896978 14063 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.974023 14046 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:56.987430 14040 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.003s
I20260709 13:01:56.993462 14056 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:56.996650  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 898) took 1100 ms. Trace:
I20260709 13:01:57.025055  9445 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.025259  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:56.934789 13419 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:01:57.002813 14073 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:57.020112 14008 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.039465  8745 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:57.027701 13941 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.055416  9444 rpcz_store.cc:276] 0709 13:01:55.896787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.946549 (+ 49762us) service_pool.cc:224] Handling call
0709 13:01:55.959404 (+ 12855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.959624 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:56.031299 (+ 71675us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.995771 (+964472us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b180034a0e0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287
0709 13:01:56.996114 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.996311 (+   197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.996356 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:56.996396 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.996458 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.996588 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2631,"prepare.queue_time_us":90480,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":4045,"raft.queue_time_us":76524,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":17526,"spinlock_wait_cycles":17892480,"thread_start_us":151728,"threads_started":2}
W20260709 13:01:57.043169  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.044468  8891 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:56.987872 13827 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.046638  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.023969 14069 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:57.056378  8889 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:01:57.038048  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 173498us
I20260709 13:01:57.058218  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.037935 13933 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:01:57.034005 14066 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:57.001696 13815 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:57.060058  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74594.2us
I20260709 13:01:57.068608  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.077608  8745 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:57.058746 14040 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.081070 14077 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.033238 14045 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.080821  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.089911  8745 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:57.073926  8747 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:01:57.038632 14036 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:57.067575 13368 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.003s
I20260709 13:01:57.082604  8898 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:57.069932 13722 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:57.083146 14082 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.083621  8818 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:57.072253 14069 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:57.083941 14071 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:57.084568 13815 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.085088  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.045914 14044 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.156256  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.156966  8747 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:57.105155  8745 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:57.088490 13997 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.159181  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.159861  8744 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:01:57.152287  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2275) took 1285 ms. Trace:
I20260709 13:01:57.162133  8744 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:57.163046  8815 raft_consensus.cc:2368] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:57.163074  8745 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:57.171600  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2282) took 1264 ms. Trace:
I20260709 13:01:57.094174 13722 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:57.174667  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2281) took 1267 ms. Trace:
I20260709 13:01:57.175202  8745 rpcz_store.cc:276] 0709 13:01:55.898279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.159086 (+1260807us) service_pool.cc:224] Handling call
0709 13:01:57.159365 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.165170 (+  5805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.165420 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.172190  8744 rpcz_store.cc:276] 0709 13:01:55.898553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.160514 (+1261961us) service_pool.cc:224] Handling call
0709 13:01:57.160801 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.162676 (+  1875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.162857 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.179666  8745 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:01:57.180498  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2009) took 1185 ms. Trace:
I20260709 13:01:57.180922  8745 rpcz_store.cc:276] 0709 13:01:55.995279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.176175 (+1180896us) service_pool.cc:224] Handling call
0709 13:01:57.179218 (+  3043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.180181 (+   963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.180425 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.162179  8741 rpcz_store.cc:276] 0709 13:01:55.867165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.555185 (+688020us) service_pool.cc:224] Handling call
0709 13:01:56.583562 (+ 28377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:56.846047 (+262485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.846094 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:56.875294 (+ 29200us) log.cc:844] Serialized 52 byte log entry
0709 13:01:56.997792 (+122498us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:57.079719 (+ 81927us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.151815 (+ 72096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.151936 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.151995 (+    59us) raft_consensus.cc:1685] finished
0709 13:01:57.152012 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.152209 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81957,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":97562,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":169,"thread_start_us":419686,"threads_started":4,"wal-append.queue_time_us":162737}
I20260709 13:01:57.197048 14035 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.200133  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 893) took 1320 ms. Trace:
I20260709 13:01:57.200556  9447 rpcz_store.cc:276] 0709 13:01:55.879586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.879784 (+   198us) service_pool.cc:224] Handling call
0709 13:01:57.199937 (+1320153us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.200077 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551097088}
I20260709 13:01:57.201900 13722 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.202526  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.203338 13722 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.106881 14046 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.205797 13574 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.158713 14082 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.208570  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.209268 14082 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.210866 14066 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.212538  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.168627 14071 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.213999  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.214550 14044 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.214695 14071 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.158835 14040 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:57.215682 14085 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:57.215446 14040 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.216732 14028 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.217967  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2265) took 1435 ms. Trace:
I20260709 13:01:57.218331  8748 rpcz_store.cc:276] 0709 13:01:55.782490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.386169 (+603679us) service_pool.cc:224] Handling call
0709 13:01:56.386443 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.551775 (+165332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.217757 (+665982us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540013b0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:57.217913 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293980032}
I20260709 13:01:57.219527  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.220342  8748 raft_consensus.cc:2410] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:57.220448 14049 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.101459 13368 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.223548  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.224225 13368 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.224859  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.225046  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.225666  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:57.226107 14056 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.226689 14056 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:57.227133 14056 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:57.227926 14056 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.228868  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.230496 14063 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.230741  8746 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:57.231210  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2555) took 1187 ms. Trace:
I20260709 13:01:57.231528 14008 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.215462  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:57.232252 14059 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.371s	user 0.004s	sys 0.000s
I20260709 13:01:57.232859  8890 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.232985 14058 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.092674 14077 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.248054  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.218350 14067 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.234897 14044 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:57.235553  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.236236  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.237720  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.272029  8748 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.239431  8891 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:01:57.240137  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 190788us
I20260709 13:01:57.272684  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.242594  8745 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:57.273378  8895 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:57.239226  8896 rpcz_store.cc:276] 0709 13:01:56.043867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.043997 (+   130us) service_pool.cc:224] Handling call
0709 13:01:57.208457 (+1164460us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaeee 0000000000375939 00007f90fb41e1e0 00007f90fb4e70c4 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:01:57.231129 (+ 22672us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11405440}
I20260709 13:01:57.255828  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:57.234337  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.219774 13898 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.241057 13574 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:57.270735  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.282481  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.283352  8740 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:01:57.241964  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:57.276206  9444 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:01:57.264003  8820 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:57.278867  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.285894  8745 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:01:57.278024  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43777.5us
I20260709 13:01:57.279585  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.287326  8895 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:01:57.287707  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.288304  8893 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:57.280685  8746 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:01:57.241627 14065 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:57.288563  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46686.4us
I20260709 13:01:57.290354  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:57.291159  9449 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:57.291080  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.291782  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.288801  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.229128 14032 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.296496 14060 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:01:57.310364  8893 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:57.291479  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.271315 14044 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.275488 14073 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:57.297279  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.293080 14051 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.298713 14077 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.317603  8748 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:57.319674  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.322256  8745 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:57.284960 14021 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:57.323518  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:57.292946 13574 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:57.357597  9449 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:57.358021  9443 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:57.297770 14065 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.319455  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.299326 14074 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:57.307571 13898 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.355477  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.356134  8740 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:57.362255  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.360548  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.359016  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.370458  8821 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:57.311460 13368 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:01:57.364163  8713 leader_election.cc:400] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:57.364871  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.365401  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:57.374132  8713 leader_election.cc:403] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:01:57.366120  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.321485 14060 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:57.376451  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:57.361227  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.321827 14073 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.379343  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.380378  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.322186 14044 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.371428  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:57.364216  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.368842 13813 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.942s	user 0.010s	sys 0.003s
I20260709 13:01:57.367444 14021 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:57.362059 14028 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.384944  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.378376  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:57.378134  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.382334  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:57.380542  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.386976  8745 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:57.387516 13978 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:57.387631  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:57.388015 13978 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.388226  8745 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:01:57.362151 13958 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:01:57.391911  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.363962 14074 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:57.381865 14022 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:57.374954 14054 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.393280 14031 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:01:57.394157 14065 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.399245  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.436328 14065 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.436756 14065 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.437073  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.390021 13813 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.438323  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.438848  8820 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.439608 13813 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.358780 14070 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:01:57.431834 13869 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.443769 13869 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.444789  9440 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 13 LEARNER]: Advancing to term 15
I20260709 13:01:57.445959 13869 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.445756  8645 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 12, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 12 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:57.447502  8645 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:57.448083 14032 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.362107 14062 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:01:57.449470 14062 consensus_queue.cc:1243] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:57.379228 14060 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.450632  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2563) took 1308 ms. Trace:
I20260709 13:01:57.450858 14060 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.363543 14063 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:01:57.398684 14094 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:01:57.401489 14072 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:57.452407  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.393334 14047 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.452785 14072 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.330483 13997 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:57.454921 13939 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.456138  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.457403  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.457399  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.458109  8824 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 9.
I20260709 13:01:57.458662  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.459250  8824 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:57.381412 14051 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:01:57.460012  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.461321  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:01:57.461702  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2286) took 1465 ms (client timeout 1905 ms). Trace:
I20260709 13:01:57.462414  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.463264  8894 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:57.464331  8894 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:57.464628  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.429116 14022 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.465631 14022 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.403949 13574 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.378903 13815 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:57.420264  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 892) took 1098 ms. Trace:
I20260709 13:01:57.468601  8741 rpcz_store.cc:276] 0709 13:01:56.322114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.185967 (+863853us) service_pool.cc:224] Handling call
0709 13:01:57.238920 (+ 52953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.239397 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.239440 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.243115 (+  3675us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.373623 (+130508us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:57.373658 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.373723 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.420031 (+ 46308us) raft_consensus.cc:1685] finished
0709 13:01:57.420059 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.420197 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":809,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4882,"spinlock_wait_cycles":11008,"thread_start_us":94288,"threads_started":1,"wal-append.queue_time_us":129210}
I20260709 13:01:57.471071  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.471331 14021 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.472478  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.468572 13815 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.473758 13815 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.381793 14073 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.381791 13733 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:57.475261 13733 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.475656 13733 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.390674  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.381699 14071 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.430583  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.484534  8822 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:57.485795  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:57.488406  8824 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:57.433112  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.368374 14077 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:57.434964 14008 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:01:57.435760  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.435995 13810 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:01:57.449528  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.497320  8896 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:57.392747  8821 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:57.451068  8897 rpcz_store.cc:276] 0709 13:01:56.142147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.142314 (+   167us) service_pool.cc:224] Handling call
0709 13:01:57.450446 (+1308132us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b5400248454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.450573 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134147456}
I20260709 13:01:57.388435 13978 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.503057  9445 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:57.503108  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.489734 13709 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.467892 14067 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:57.505573 13810 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:57.496765 14032 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.489508  8741 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:57.494017  8891 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:57.507997  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.508019 14076 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.508620  8741 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:57.508755 14076 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.509238 14076 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.429643 14074 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.449210  8645 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 10, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 10 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:57.430716  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.491858  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:57.491879  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.495986  8820 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:01:57.501571 13783 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:57.506502 13810 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.509560  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:57.513092  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82591.4us
W20260709 13:01:57.462136  8742 rpcz_store.cc:269] 0709 13:01:55.995858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.083995 (+ 88137us) service_pool.cc:224] Handling call
0709 13:01:57.461516 (+1377521us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54003bfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.461619 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52418816}
I20260709 13:01:57.513589  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:57.390024  8745 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:57.514727  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.523640  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.524370  8820 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:57.511166  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.515282  8821 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:57.530647  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.562237  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:57.562678  9440 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 15 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:57.530510 14032 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:57.564180  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.567837  9449 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:01:57.525035 14008 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:57.566555  8742 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:01:57.568859  8741 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:57.572664  8820 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:57.573987  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.575700  8742 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:57.564630  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 129986us
I20260709 13:01:57.588642  9440 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:57.561980 13709 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:01:57.592471 13839 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.592989 13839 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.561758 14028 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:57.596196  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.598136  8820 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:57.514810 13958 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:01:57.602237 13869 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:57.602118 13722 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:01:57.566895 14046 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.603920  8742 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:57.567517 14032 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.575562 14062 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:57.617619  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.590834 14070 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.625244  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.562439 13898 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:01:57.608817  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.629268  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.590693 13810 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.581331 13933 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.516587 14073 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.613936  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:57.597084 14008 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:57.579166  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57416.4us
I20260709 13:01:57.623826 14110 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:57.600419 14028 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.598560 14058 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:01:57.590540 14094 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:57.610427 14046 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.612664 13917 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:57.616402 14032 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.625748 14048 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:57.619623 14062 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.603907 14069 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.651973 13898 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:01:57.625906  8893 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:01:57.609799 13815 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.649858 14028 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:57.648142 14072 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.677066 14046 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.682165 14062 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.645757 14070 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:57.700908  8823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 867) took 1939 ms (client timeout 1550 ms). Trace:
W20260709 13:01:57.699369  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101674us
W20260709 13:01:57.701812  8823 rpcz_store.cc:269] 0709 13:01:55.779677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.814327 (+ 34650us) service_pool.cc:224] Handling call
0709 13:01:57.700676 (+1886349us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54001f04d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.700809 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155826432}
I20260709 13:01:57.678179 13917 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:57.695070  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 109567us
I20260709 13:01:57.645588 13933 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.707139  8896 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
W20260709 13:01:57.696251 14097 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:01:57.695474 13898 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:57.692875 14056 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:57.723426  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84999.1us
I20260709 13:01:57.697348 13929 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.682756 14048 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.729475  8823 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:57.730296 14056 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.728332 13722 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.735244 13722 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.736351 14056 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.723918 13898 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:57.741693 14058 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.756860  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50860.5us
I20260709 13:01:57.742375 14069 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.723717 14008 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.765040 13839 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.787587 13815 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.788336 14073 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.789088 14073 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:57.791265 14073 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:57.792809 14073 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:57.794240 14070 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.792322 13933 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.795174 14073 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.797340 14073 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:57.797570  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.745272 13917 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.766605 14058 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.821336  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:57.825974  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.828754  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.833843  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:57.834424  8893 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.836863 13917 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:57.838927 13929 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:57.839349 13929 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.839718 13929 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.826983 14069 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:57.819561 14038 raft_consensus.cc:476] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Not starting pre-election -- already a leader
I20260709 13:01:57.845942 14065 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.846235 14060 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.848093 14060 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:57.848170 14065 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.841023 14071 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:57.849938 13978 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.852092 13978 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.833065 14094 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:57.854415 14094 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.854765 14094 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:57.786257 13898 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.858422 13898 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.824584 14070 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.861697 14070 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.864619  8824 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:01:57.865810  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.866914  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:01:57.871068 14104 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.248s	user 0.002s	sys 0.000s
I20260709 13:01:57.878163  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:57.879392  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.880395  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.882442  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.883663  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.884195  8748 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:57.886701 14058 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.889215 13810 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:57.889771  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:57.890475  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2637) took 1275 ms. Trace:
I20260709 13:01:57.890920  8815 rpcz_store.cc:276] 0709 13:01:56.615040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.615194 (+   154us) service_pool.cc:224] Handling call
0709 13:01:57.890278 (+1275084us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540012b954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.890411 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576569216}
I20260709 13:01:57.893565  8748 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.894114  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2347) took 1008 ms. Trace:
W20260709 13:01:57.900110  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.901904  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 984) took 1234 ms. Trace:
I20260709 13:01:57.902441  9446 rpcz_store.cc:276] 0709 13:01:56.667440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.713274 (+ 45834us) service_pool.cc:224] Handling call
0709 13:01:57.901682 (+1188408us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54001982d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.901827 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322987776}
W20260709 13:01:57.903920  8749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2008) took 1909 ms (client timeout 1734 ms). Trace:
W20260709 13:01:57.904440  8749 rpcz_store.cc:269] 0709 13:01:55.994953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.017682 (+ 22729us) service_pool.cc:224] Handling call
0709 13:01:57.903680 (+1885998us) spinlock_profiling.cc:243] Waited 77.6 ms on lock 0x7b54004669d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:57.903823 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178397568}
I20260709 13:01:57.905786  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.907815  8749 pending_rounds.cc:85] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 6
I20260709 13:01:57.908171  8749 pending_rounds.cc:107] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:57.908485  8749 raft_consensus.cc:2889] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:57.911212 13722 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.911865  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:57.912736 14040 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.914386  8819 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:57.915684 13535 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.916265 13862 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:57.916321 13535 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:57.917744 13912 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.926222  8740 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:57.926718 14045 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:57.862239  8822 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:01:57.877702  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:57.938802 14005 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:57.939368 14005 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:57.940973 14044 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.007s
I20260709 13:01:57.942721 14054 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:57.944187  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 56626.4us
W20260709 13:01:57.949738  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.949681  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49726.2us
W20260709 13:01:57.950336  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.889860 13810 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:57.917187 13862 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:57.959698  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 339010us
I20260709 13:01:57.984719  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:57.970096  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:57.971972  8816 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:57.978128  8819 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:57.986436 13941 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.970188  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.987701  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2632) took 1378 ms. Trace:
I20260709 13:01:57.980123  9445 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:57.988077  8818 rpcz_store.cc:276] 0709 13:01:56.609541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.609689 (+   148us) service_pool.cc:224] Handling call
0709 13:01:56.886301 (+276612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.099949 (+213648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.987472 (+887523us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:57.987619 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453029760}
I20260709 13:01:57.988742 14122 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.988904  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.982649  8893 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:57.992187  8748 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:01:57.894621  8892 rpcz_store.cc:276] 0709 13:01:56.885935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.890668 (+  4733us) service_pool.cc:224] Handling call
0709 13:01:56.890929 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.156859 (+265930us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:57.269368 (+112509us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.369626 (+100258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.369672 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:57.369719 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.892925 (+523206us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b180028ae00. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419
0709 13:01:57.893671 (+   746us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540045a6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:57.893720 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.894040 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60532,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":91,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4359,"spinlock_wait_cycles":613868160,"thread_start_us":211198,"threads_started":2,"wal-append.queue_time_us":114005}
I20260709 13:01:58.007226  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.012682  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.001266  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:57.997267  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.016383  9445 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:58.059597  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.060298  8748 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.989521 14063 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:57.974380 14054 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.980908 14048 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:57.940564 14005 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.087738  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:58.090209  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 92369.3us
W20260709 13:01:58.088243  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 97520.5us
W20260709 13:01:58.093737  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 81170.4us
I20260709 13:01:58.114693  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:58.122438  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.122931 13535 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.123571  9446 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:58.124768  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:57.996788 13646 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:01:58.004763 14120 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.137698  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.131508  8748 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:58.066241 13898 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.067204 13941 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:58.157253  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1030) took 2102 ms. Trace:
I20260709 13:01:58.157801  9441 rpcz_store.cc:276] 0709 13:01:56.044971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.045144 (+   173us) service_pool.cc:224] Handling call
0709 13:01:56.069039 (+ 23895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.898333 (+829294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.147451 (+1249118us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54006a8954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:58.147683 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873617152}
I20260709 13:01:58.158914  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 894) took 1836 ms. Trace:
I20260709 13:01:58.088167 14063 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:58.159375  8743 rpcz_store.cc:276] 0709 13:01:56.322670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.227241 (+904571us) service_pool.cc:224] Handling call
0709 13:01:57.227493 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.227688 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:57.243786 (+ 16098us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.945530 (+701744us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b1800345700. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:01:58.158598 (+213068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.158650 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:58.158692 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.158755 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.158866 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15551,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"spinlock_wait_cycles":9245056,"thread_start_us":212859,"threads_started":1}
I20260709 13:01:58.014775 14115 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:01:58.166893  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.014915 14032 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.117949 14122 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:58.095000 14022 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 13:01:58.069710 14070 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.156853 13898 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.061435 14118 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:57.971921 14044 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.167704  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.175462  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.065129 14075 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:01:58.135581 14119 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:58.192960 14119 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:01:58.194378 13733 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.005s
I20260709 13:01:58.064395 13912 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.194923 13733 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.195073 13912 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.180974  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.185197  8748 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:58.197453  8823 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:58.186877  9449 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:58.157035 13535 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:58.201550  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.203110  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.204370  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.191618  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.191730  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:58.153713 13929 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.206733 13827 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.206820  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:58.207211 13839 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.207736 13839 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:58.208074 13827 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:58.208572 13839 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.209681 14065 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:58.211369  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.213930  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.197042 13733 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.215749 13917 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:58.216914 14124 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.204975  9449 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:01:58.217540 14124 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:58.218680 14124 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.220619 14071 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:58.219956  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.221670  9441 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:58.216712  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 875) took 1280 ms. Trace:
I20260709 13:01:58.222635  8822 rpcz_store.cc:276] 0709 13:01:56.935861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.526773 (+590912us) service_pool.cc:224] Handling call
0709 13:01:57.566371 (+ 39598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.936775 (+370404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.216462 (+279687us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540024a9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:58.216654 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679970560}
I20260709 13:01:58.223500  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.201815 13535 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.220986 14071 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:58.225085 13535 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:58.225593 14071 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.226251  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.220201 14036 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.226926  8746 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:01:58.181181 13815 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:58.217480  8742 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:58.229498  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.229882  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 878) took 1284 ms. Trace:
I20260709 13:01:58.193949 14119 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:58.230019  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.230311  8822 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:58.172420 13722 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.191910 13862 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.229416  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.255084  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:58.246286 14049 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:58.266726  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.267000 14035 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.267663 14035 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.267237 14049 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.267968  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.268613  8893 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:58.269696 14049 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.264622  8824 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13. Re-sending same reply.
I20260709 13:01:58.230322  8821 rpcz_store.cc:276] 0709 13:01:56.945568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.530009 (+584441us) service_pool.cc:224] Handling call
0709 13:01:57.530285 (+   276us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:57.674873 (+144588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.674919 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.695670 (+ 20751us) log.cc:844] Serialized 52 byte log entry
0709 13:01:57.784942 (+ 89272us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:57.784985 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.785060 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.229677 (+444617us) raft_consensus.cc:1685] finished
0709 13:01:58.229697 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.229827 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26126,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":112635,"threads_started":2,"wal-append.queue_time_us":428132}
I20260709 13:01:58.216562 14069 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.264248  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.188457  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.238395  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.283057  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:58.230037  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2280) took 2332 ms. Trace:
I20260709 13:01:58.232805  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.232329  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.168612 13941 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.216614 13917 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:58.181409 14022 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.185734 13898 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:58.259315  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.244969  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:58.258006  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:01:58.258823  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.262914  9449 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:58.226816 14044 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.229557 14066 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:01:58.155036 14120 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.248553 14075 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:01:58.268694 14035 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.233991  8818 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:58.306591  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.390414  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.284937  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:01:58.257992 14076 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.748s	user 0.010s	sys 0.003s
I20260709 13:01:58.401523  8897 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.403404  8823 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15.
W20260709 13:01:58.255825 14074 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.000s
I20260709 13:01:58.402390  8821 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:58.229920 14116 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.199s
I20260709 13:01:58.431097  9440 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:58.315248 14032 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:58.307160 14046 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
W20260709 13:01:58.357669  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.429742  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.304560 14047 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:58.431479 14074 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.271296 13722 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.436796  8817 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:58.226843 14071 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.441151  8743 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:01:58.239491 14028 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.445112  8747 rpcz_store.cc:276] 0709 13:01:55.898015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.158529 (+1260514us) service_pool.cc:224] Handling call
0709 13:01:57.173787 (+ 15258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.229805 (+ 56018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.229834 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.230287 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.372129 (+141842us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:58.229559 (+857430us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.229759 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.229826 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.229866 (+    40us) raft_consensus.cc:1685] finished
0709 13:01:58.229877 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.229986 (+   109us) inbound_call.cc:177] Queueing success response
0709 13:01:58.248148 (+ 18162us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b18003b20c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
Metrics: {"mutex_wait_us":870324,"prepare.queue_time_us":865554,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":5563,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":197,"spinlock_wait_cycles":17642880,"thread_start_us":137246,"threads_started":1,"wal-append.queue_time_us":154356}
I20260709 13:01:58.388626 14122 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:58.467554 14074 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.469519  8748 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:01:58.470191  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:58.306135 14063 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.487253 14063 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.487516  9446 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:58.495595  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2303) took 1007 ms. Trace:
I20260709 13:01:58.496021  8740 rpcz_store.cc:276] 0709 13:01:57.488307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.928788 (+440481us) service_pool.cc:224] Handling call
0709 13:01:57.929236 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.015821 (+ 86585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.015866 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.017493 (+  1627us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.264812 (+247319us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:58.330797 (+ 65985us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.331052 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.331156 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.495372 (+164216us) raft_consensus.cc:1685] finished
0709 13:01:58.495396 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.495533 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58665,"prepare.queue_time_us":105621,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":58748,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7470,"thread_start_us":138632,"threads_started":1,"wal-append.queue_time_us":300977}
W20260709 13:01:58.498221 14062 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.779s	user 0.014s	sys 0.000s
I20260709 13:01:58.498710 14062 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.458585 14046 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.503623 14046 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.273733 13535 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.506157 13535 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:58.508565 14094 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.653s	user 0.006s	sys 0.005s
I20260709 13:01:58.510435 14022 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.430871 14076 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.515043 14076 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.523236  8748 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:01:58.523603 13912 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.462275 14122 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.530253 13722 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.509160 14094 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.535456 14094 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.535647  9443 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:01:58.540220  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2353) took 1652 ms (client timeout 1523 ms). Trace:
I20260709 13:01:58.305750 14069 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.158s
I20260709 13:01:58.433638 14024 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:58.457894 13578 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:01:58.548341  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 313975us
I20260709 13:01:58.548681  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2299) took 1062 ms. Trace:
W20260709 13:01:58.540864  8898 rpcz_store.cc:269] 0709 13:01:56.887730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.888106 (+   376us) service_pool.cc:224] Handling call
0709 13:01:58.539954 (+1651848us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400047ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:58.540130 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387504000}
I20260709 13:01:58.316454  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:58.560788  8824 raft_consensus.cc:606] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:58.480867  8897 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:01:58.241863 13827 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:58.567114  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.544833  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.568467 13733 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.568866 13733 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:58.559499 13578 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:58.569247 13733 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.569970 13578 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.571171 13578 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:58.540520 13839 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.550877 14062 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.579198  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.575438  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.600649  8816 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.598230  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.605899  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.559948 13722 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.554226  8745 rpcz_store.cc:276] 0709 13:01:57.486455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.579630 (+ 93175us) service_pool.cc:224] Handling call
0709 13:01:57.579904 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.580081 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:58.230279 (+650198us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.546017 (+315738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.546084 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:58.548426 (+  2342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.548512 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.548629 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":649905,"prepare.queue_time_us":658222,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"thread_start_us":315348,"threads_started":1,"wal-append.queue_time_us":315711}
I20260709 13:01:58.629977  8745 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:01:58.632011  8745 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:58.569038 14120 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:58.622866  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 116471us
I20260709 13:01:58.639849  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:58.609906  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.566357 14073 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:01:58.650933 13941 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.647714  8748 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:58.644021 14153 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:58.645503 14048 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.652710  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2073) took 1135 ms. Trace:
W20260709 13:01:58.653185  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.653918  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:58.658601  8824 raft_consensus.cc:606] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:58.658980  8824 raft_consensus.cc:614] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.615454 14038 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.634745 14069 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:58.664445 14069 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:01:58.652688 14054 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
I20260709 13:01:58.665540 14054 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.666138 14073 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:58.666530 14073 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.667408 14073 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.668573 14073 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.636978 13722 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.670557 14063 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:01:58.671286 14063 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:58.671876 13722 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.657159  8820 rpcz_store.cc:276] 0709 13:01:57.517110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.607685 (+ 90575us) service_pool.cc:224] Handling call
0709 13:01:57.607955 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.728092 (+120137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.728120 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.728159 (+    39us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:01:57.728187 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.652501 (+924314us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003e3fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:58.652544 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.652657 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275896704}
I20260709 13:01:58.663291 14028 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.639374 14008 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:01:58.673883 14153 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:58.674381 14153 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:58.672422 14063 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:58.675227 14153 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.667632 14054 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:01:58.677178  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.564680 14024 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:58.658478  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48631.1us
I20260709 13:01:58.678006 14038 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.682049  8715 leader_election.cc:400] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15.
W20260709 13:01:58.688007 14102 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 1.107s	user 0.005s	sys 0.000s
I20260709 13:01:58.677834 14048 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.652648 13978 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.692991  8824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:01:58.696573  8824 raft_consensus.cc:606] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:58.696930  8824 raft_consensus.cc:614] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.695245  8715 leader_election.cc:403] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:58.627012 13839 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.705794  8816 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
W20260709 13:01:58.726711  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44774.7us
I20260709 13:01:58.702250  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 13:01:58.731964  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 106082us
I20260709 13:01:58.733165  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2300) took 1245 ms. Trace:
I20260709 13:01:58.738353  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:01:58.742820 14106 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.122s	user 0.000s	sys 0.003s
I20260709 13:01:58.764981  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2033) took 1025 ms. Trace:
I20260709 13:01:58.702389 14053 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.747325 14048 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.751202 13898 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.752434 13733 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.703066 13978 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.757915 14024 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:01:58.771849  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.728964 13917 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15.
I20260709 13:01:58.681586 14122 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.675969 14063 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.729206  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.766212 14053 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:58.734135  8741 rpcz_store.cc:276] 0709 13:01:57.487244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.582214 (+ 94970us) service_pool.cc:224] Handling call
0709 13:01:57.582480 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.582647 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:58.620538 (+1037891us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.731993 (+111455us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.732071 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:58.732138 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.732242 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.733034 (+   792us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647800,"prepare.queue_time_us":670872,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":223,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":497457,"threads_started":2}
I20260709 13:01:58.765404  8740 rpcz_store.cc:276] 0709 13:01:57.739397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.497704 (+758307us) service_pool.cc:224] Handling call
0709 13:01:58.498026 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.730077 (+232051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.730187 (+   110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.764697 (+ 34510us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:58.764740 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.764813 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.764929 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37783,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":437,"thread_start_us":231420,"threads_started":1}
I20260709 13:01:58.762574 14038 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.762574 13839 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.775000  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.696116 13827 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:58.781457  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 52369.4us
I20260709 13:01:58.772049 14123 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:58.789927  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:58.792865  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.787068 14152 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.794164 14054 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.801595  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:58.804760 13978 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.808779  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.809374  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.810770  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.811385  8745 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:58.811985  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.812476  8746 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.813071  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.813687  8740 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.803185 14157 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:58.821262  8897 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:58.821431  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2360) took 1904 ms. Trace:
I20260709 13:01:58.821943  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.821837  8890 rpcz_store.cc:276] 0709 13:01:56.914588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.074668 (+160080us) service_pool.cc:224] Handling call
0709 13:01:57.091367 (+ 16699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.245463 (+154096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.818299 (+1572836us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540046e954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:58.818472 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325012736}
I20260709 13:01:58.823289  8740 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:58.798810 13917 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:01:58.842427  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2305) took 1328 ms. Trace:
I20260709 13:01:58.842917  8743 rpcz_store.cc:276] 0709 13:01:57.513803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.459333 (+945530us) service_pool.cc:224] Handling call
0709 13:01:58.460205 (+   872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.700850 (+240645us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:58.700890 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.727832 (+ 26942us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.795014 (+ 67182us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:58.795053 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.795760 (+   707us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.808138 (+ 12378us) raft_consensus.cc:1685] finished
0709 13:01:58.808158 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.842369 (+ 34211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63830,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":136,"thread_start_us":295609,"threads_started":2,"wal-append.queue_time_us":64852}
I20260709 13:01:58.825788  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.826880  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.845431  8898 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:01:58.802031 14152 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:58.847057  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.836967 14047 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:01:58.841087 14146 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:58.800211 14147 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.844529  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.844528  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:58.845527  8897 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:58.831823  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 715355us
I20260709 13:01:58.864796  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.828814 13810 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.807437 14123 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.879926  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.881028  9444 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:58.882702  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.886113  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.890049  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:58.891321  9443 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:58.819249 14054 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:58.865761  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:58.865777  8890 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:01:58.847919  8712 connection.cc:256] RPC call timeout handler was delayed by -1.4134819489999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 13:01:58.900803  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3866927189999387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:01:58.869195  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:58.901367  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3570356089999223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:01:58.901821  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3387819489998947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
I20260709 13:01:58.901976  8743 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:58.902307  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.903136  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2118819489999169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:01:58.903604  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2091819489999125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
I20260709 13:01:58.846285 13912 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.848645 14047 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:58.907590  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:58.820362 14048 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:58.908125  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.915624  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.916639  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.917023  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:58.917269  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49403.9us
I20260709 13:01:58.917656  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.917659  8743 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:01:58.918251  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.918567  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.919809  8893 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:01:58.919780  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.920269  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:58.908344  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.909180  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:58.903995  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1389819489999127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:01:58.921233  8898 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 11: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:58.848901 14152 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.912889  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.928977  8891 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.925503  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.929085  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.913251  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:58.913679  9444 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:58.892560 14115 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:58.922425  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.872771 14157 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.923363  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.923774  8892 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:58.924350  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.937093  8740 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.924944  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.925598  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.926477  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.938108 14048 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.938225 14126 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:58.932579  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.939267  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.939189 14048 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.939828  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.940460  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
W20260709 13:01:58.947000 14132 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.398s	user 0.001s	sys 0.001s
I20260709 13:01:58.952896  8740 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:58.933419  9443 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.914052  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.935083  8898 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:58.938308  8741 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:58.873697 14147 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.966580  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.972723 14054 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:58.972985  8820 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.929692 14021 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:58.973662 14021 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:58.974054 14021 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.976408 14005 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.978425 14051 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:01:58.978940 14137 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:01:58.979027 14051 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.979820  8820 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:01:58.980585  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 891) took 2658 ms. Trace:
I20260709 13:01:58.913329 13978 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:01:58.981873  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.980978  8744 rpcz_store.cc:276] 0709 13:01:56.321776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.185319 (+863543us) service_pool.cc:224] Handling call
0709 13:01:57.240085 (+ 54766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.245185 (+  5100us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:58.223197 (+978012us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18003456a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:01:58.223518 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.547643 (+324125us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.553443 (+  5800us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.553630 (+   187us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:58.553681 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.553774 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.977782 (+424008us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18003456a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db
0709 13:01:58.978417 (+   635us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:58.980385 (+  1968us) raft_consensus.cc:1685] finished
0709 13:01:58.980409 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.980529 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5621,"prepare.queue_time_us":971509,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":30884,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":28301,"spinlock_wait_cycles":107822592,"thread_start_us":323121,"threads_started":1,"wal-append.queue_time_us":323485}
I20260709 13:01:58.912835 14073 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.982728 14120 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:58.983103  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.983367 14073 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.984146  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.984202 13941 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:58.915817 14031 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:01:58.954051  8890 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:58.985057 13941 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:58.985443 14031 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:58.920874 13733 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.985729  8744 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:01:58.986514 13941 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:58.957969 14158 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:01:58.972532 14069 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.913343 13839 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:58.959287 14143 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:01:58.867396 13917 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:58.966838  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:59.015777  8823 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.913414 13961 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:58.967211  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.967547  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:58.968261  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:58.969260  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:58.953887  8746 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:58.970888  8645 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.989713  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:58.971539  9443 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:01:58.972214  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:58.961262  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:58.992663  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.968113  8818 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:58.993901  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.994428  8743 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:01:58.970045  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2302) took 1482 ms. Trace:
I20260709 13:01:59.002064  8715 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:58.927991  8897 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
W20260709 13:01:59.005488  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 78508.5us
I20260709 13:01:59.007283  8745 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:01:59.009500  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:58.970953  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:58.921212  9447 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:01:59.010076  8894 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:59.025774 14155 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:01:58.971186 14147 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:58.990341 14115 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:59.026562 14115 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:59.026659  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.027796 13929 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:58.929785 14028 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.019050 13961 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:59.018270  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:58.896611 14045 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:01:59.010119 13978 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:59.030766 13978 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.033210  8815 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 10: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:58.965977 14076 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:01:59.034310  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 73127.1us
I20260709 13:01:58.916021 13912 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.022456  8749 rpcz_store.cc:276] 0709 13:01:57.488025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.907125 (+419100us) service_pool.cc:224] Handling call
0709 13:01:57.907558 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:57.976803 (+ 69245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.976847 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.977599 (+   752us) log.cc:844] Serialized 52 byte log entry
0709 13:01:58.438822 (+461223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:58.438871 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.438939 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.946429 (+507490us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:58.969765 (+ 23336us) raft_consensus.cc:1685] finished
0709 13:01:58.969787 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.969986 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67525,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":17159,"spinlock_wait_cycles":13294976,"thread_start_us":323228,"threads_started":1,"wal-append.queue_time_us":499115}
I20260709 13:01:59.019680  8816 raft_consensus.cc:2368] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:59.035080  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:59.036782  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.049933  8746 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 10: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:59.038980  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.019675  8741 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.038709  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.041812  8823 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:59.049678  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.052927  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:59.053159  9447 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:59.017058 13941 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.060463  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.037074 14069 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:59.068835  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79190.7us
I20260709 13:01:59.069522  9444 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:01:59.019073 14122 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.067593  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:59.073390 14159 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.038048 14076 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:59.078594  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:59.041318 13929 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:59.077699 14151 raft_consensus.cc:2764] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:01:58.984577 14073 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.017357 13839 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.082911  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.085330  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:59.105409  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2032) took 1366 ms (client timeout 1555 ms). Trace:
W20260709 13:01:59.105968  8742 rpcz_store.cc:269] 0709 13:01:57.739152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.877425 (+138273us) service_pool.cc:224] Handling call
0709 13:01:59.101179 (+1223754us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400358c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:59.105339 (+  4160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644729088}
I20260709 13:01:59.107687  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.108518  8742 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.081043 14062 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:59.087600  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.059536 14147 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.088430  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:59.133420  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:59.090791 14149 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:59.093674 14045 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:59.087451 13765 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:59.097157 14067 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:59.078389 14051 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:59.098213  8642 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 285) took 31001 ms (client timeout 15000 ms). Trace:
I20260709 13:01:59.098416  8645 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 3 to 9, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.102017  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 910) took 1086 ms. Trace:
I20260709 13:01:59.060575 13709 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.111492  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:59.073199 13958 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.073352 14033 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.030194 13961 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.110800  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:01:59.133108  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 47865.7us
I20260709 13:01:59.136859  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:59.141201  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52862us
I20260709 13:01:59.059943 13898 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.066154 14153 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:01:59.036226 14123 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.087541 14094 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
W20260709 13:01:59.338609 14163 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:01:59.343314 14054 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:59.343488 14047 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.344125 14054 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:59.344624  9448 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:01:59.339753 14110 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:59.345364 14110 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.345198 14054 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.307587  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:59.325699  9444 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:59.346536  8746 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:01:59.291369 14156 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.375s	user 0.002s	sys 0.000s
I20260709 13:01:59.347481 14110 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.295176 13839 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:59.336951 14165 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.243s	user 0.002s	sys 0.000s
I20260709 13:01:59.309592 13917 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.348431  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 912) took 1260 ms. Trace:
I20260709 13:01:59.349340 13958 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.349826  9442 rpcz_store.cc:276] 0709 13:01:58.087917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.486420 (+398503us) service_pool.cc:224] Handling call
0709 13:01:58.486820 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:58.559553 (+ 72733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.559601 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:58.561685 (+  2084us) log.cc:844] Serialized 52 byte log entry
0709 13:01:58.790017 (+228332us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:58.790056 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.790122 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.290569 (+500447us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:01:59.348162 (+ 57593us) raft_consensus.cc:1685] finished
0709 13:01:59.348188 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.348384 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62516,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":13081,"spinlock_wait_cycles":3698432,"thread_start_us":163165,"threads_started":1,"wal-append.queue_time_us":282216}
I20260709 13:01:59.351521 13815 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:59.351902 13958 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.352151 14115 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.352574 13815 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:59.140720 14151 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.353114 13815 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:01:59.196761  8741 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:59.354089 13815 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:59.316145  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 230151us
I20260709 13:01:59.293952  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2367) took 1673 ms. Trace:
W20260709 13:01:59.357398  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 224112us
I20260709 13:01:59.307217  8747 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:01:59.329200  8742 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:01:59.298810  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 873) took 2363 ms. Trace:
I20260709 13:01:59.365324  8891 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:01:59.365525  8817 rpcz_store.cc:276] 0709 13:01:56.935339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.506543 (+571204us) service_pool.cc:224] Handling call
0709 13:01:57.506790 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.437279 (+930489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.298555 (+861276us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b5400075654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:59.298741 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138546944}
I20260709 13:01:59.302966  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:01:59.366942  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64042.8us
I20260709 13:01:59.367987  8715 leader_election.cc:400] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:01:59.368629  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.367671  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:01:59.151531  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64015.5us
I20260709 13:01:59.370644  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:59.332294  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.356772  8740 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:59.355748  9441 rpcz_store.cc:276] 0709 13:01:58.015455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.166990 (+151535us) service_pool.cc:224] Handling call
0709 13:01:59.101809 (+934819us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:01:59.101956 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55632256}
I20260709 13:01:59.357580  8896 rpcz_store.cc:276] 0709 13:01:57.620461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.644939 (+ 24478us) service_pool.cc:224] Handling call
0709 13:01:57.670372 (+ 25433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.711220 (+ 40848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.293678 (+1582458us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b540046ee54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:01:59.293885 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84042112}
W20260709 13:01:59.229833  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 141422us
I20260709 13:01:59.114212 14062 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:59.125886 14121 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:59.377080  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.359375  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:01:59.190042 14161 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:59.135380 14074 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.151257 14116 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:59.294749 14139 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.748s	user 0.000s	sys 0.003s
I20260709 13:01:59.407895  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:59.151520 13578 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.354689 14115 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:59.460793  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53038.9us
W20260709 13:01:59.442351  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 83054.5us
W20260709 13:01:59.463325  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 88145.1us
I20260709 13:01:59.469852  8740 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
W20260709 13:01:59.455013  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96414.7us
W20260709 13:01:59.455976  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 87415us
I20260709 13:01:59.330973 14120 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.378453 13862 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:59.138485 14148 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:59.179200 14028 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.452337  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:59.379417  8645 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:01:59.444290  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:59.359905 14045 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.199625 13912 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.489502  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:59.287839 13978 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.473028  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.493913  8817 raft_consensus.cc:2393] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:59.411645  8824 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:59.494284  8815 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:59.479264  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:59.462548 14022 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:59.383816 13815 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:59.376624 14151 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:59.484571  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.481427 14164 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.487269  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.383883 13813 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:59.454212 13869 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:59.505630 13869 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:59.506078 13869 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:59.506255 14152 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.000s
I20260709 13:01:59.506687 14152 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.508790 14152 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:01:59.509439  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.510325 14047 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.497786 14028 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.511525  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:01:59.462409 14036 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:01:59.197095 14021 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.515305 14021 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.460353 14116 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.516152 14116 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.378683 14033 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.458954 13419 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:59.517668 13419 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:59.518083 13419 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.518926 14033 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.493584 14148 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.410770 13839 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 13:01:59.384287 14062 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.521813 14148 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.488967 14120 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.522603  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:59.524435 14120 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.493088 14035 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:59.525429 14035 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.526454 13419 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.527997 14035 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.528784 13419 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:59.529902 14005 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.003s
I20260709 13:01:59.180306 14149 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.531764 14116 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.532440 14149 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.533190  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:01:59.534688 14116 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:59.534564 14164 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.536252 14062 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.536617 13733 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:59.537251 13733 consensus_queue.cc:1243] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:59.538471 14062 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.539250 13733 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.540042 13933 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.541776  8816 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:01:59.541924 14036 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:59.542363 14036 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:59.533246 14022 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:01:59.206557 13765 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.501760  8645 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 10, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.512980 14028 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.519176 13869 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.286880 14053 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:01:59.552707  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 73561.8us
I20260709 13:01:59.454073 13722 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:01:59.558698  8645 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 8, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:59.559841  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 477098us
I20260709 13:01:59.563256  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:59.356791  8642 rpcz_store.cc:269] 0709 13:01:28.096878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.109432 (+ 12554us) service_pool.cc:224] Handling call
0709 13:01:59.097958 (+30988526us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:51.024275 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:51.220123 (+195848us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:51.230151 (+ 10028us) write_op.cc:435] Acquired schema lock
0709 13:01:51.231923 (+  1772us) tablet.cc:662] Decoding operations
0709 13:01:51.304695 (+ 72772us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:51.304807 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:51.304834 (+    27us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:01:51.316735 (+ 11901us) tablet.cc:701] Row locks acquired
0709 13:01:51.316785 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:01:51.316971 (+   186us) write_op.cc:270] Start()
0709 13:01:51.317175 (+   204us) write_op.cc:276] Timestamp: P: 1783602111316903 usec, L: 0
0709 13:01:51.317213 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:01:51.401833 (+ 84620us) log.cc:844] Serialized 5173 byte log entry
0709 13:01:52.200572 (+798739us) op_driver.cc:464] REPLICATION: finished
0709 13:01:52.461412 (+260840us) write_op.cc:301] APPLY: starting
0709 13:01:52.461589 (+   177us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:52.589833 (+128244us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:52.607081 (+ 17248us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:52.898033 (+290952us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:52.898397 (+   364us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:01:52.898458 (+    61us) write_op.cc:312] APPLY: finished
0709 13:01:52.898852 (+   394us) log.cc:844] Serialized 121 byte log entry
0709 13:01:52.899041 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:52.899244 (+   203us) write_op.cc:454] Released schema lock
0709 13:01:52.899313 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:52.948053 (+ 48740us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:52.952733 (+  4680us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1586431,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":208693,"apply.run_cpu_time_us":23984,"apply.run_wall_time_us":496162,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58441,"num_ops":13,"prepare.queue_time_us":365455,"prepare.run_cpu_time_us":45134,"prepare.run_wall_time_us":600016,"raft.queue_time_us":352717,"raft.run_cpu_time_us":13269,"raft.run_wall_time_us":272858,"replication_time_us":862226,"thread_start_us":737415,"threads_started":4,"wal-append.queue_time_us":187906,"wal-append.run_cpu_time_us":93795,"wal-append.run_wall_time_us":2375749}]]}
I20260709 13:01:59.564540  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.566234  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.492888 14045 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.384532 14159 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.172s
I20260709 13:01:59.571056  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.571230  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.580088  8891 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:59.584990  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:59.585440  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.586596  8893 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:59.589270  8645 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.557401 14115 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:59.591982  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.598623  8893 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:59.537278 14164 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:01:59.594398  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.603544  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.566488 14122 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:01:59.595824  8892 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:01:59.594424  9446 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:01:59.558178 14035 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:01:59.602586  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:59.554981 14151 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:01:59.604665  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:59.607026  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.621675 14075 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:59.622902  8645 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.623863  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:59.572525 14149 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:59.624842  8645 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 8, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.582531 14148 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:59.627653  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.633770  8889 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:59.634606  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.625082  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:01:59.595012  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 490038us
I20260709 13:01:59.563622 14152 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:01:59.638674  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:01:59.638744  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.640131  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:01:59.617069 14063 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.640749  9449 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:59.640120  8892 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:01:59.598228 14115 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:01:59.617467 13898 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:59.645828  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 172890us
W20260709 13:01:59.646698  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 162224us
I20260709 13:01:59.632161 14075 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:59.641520 13941 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 13:01:59.673321  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:59.674362  8745 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:59.651875  8645 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 5 to 8, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 8 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:01:59.624791 14035 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:59.681519  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 145551us
W20260709 13:01:59.682715  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.683815  8742 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: [10.10-13.12]
I20260709 13:01:59.684924  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.685457  8741 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:01:59.687016  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:59.687433  8742 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:59.662264  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.650183  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.668593  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:01:59.691687  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 247494us
I20260709 13:01:59.692487  8891 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:01:59.697556  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47484.6us
I20260709 13:01:59.698058  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:59.698424  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.627250 13917 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:01:59.702533  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.706130  8746 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:59.702375  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:59.707167  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.708509  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:01:59.642887 13810 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:59.574059 14036 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.710425  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.704078  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:01:59.704986  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44836.6us
I20260709 13:01:59.711876  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.713105  8743 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:01:59.712846  8645 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 6 to 8, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 8 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.658005 14115 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:59.734978  9449 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:01:59.727415  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:01:59.739663  8893 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:01:59.742936  8747 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:01:59.745234  9412 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:01:59.748651  8645 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 4 to 7, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 7 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:59.713749 14145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.013s	sys 0.040s
I20260709 13:01:59.667598 14075 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:01:59.752851  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50606us
I20260709 13:01:59.808460  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.655706 14073 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 13:01:59.792102  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:01:59.713196 14051 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:59.762470 13941 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:59.763144 14069 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 13:01:59.714267 13958 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:59.724803 14021 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:59.805179  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60956.5us
I20260709 13:01:59.759084 14033 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:59.763988 14111 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:59.744336 14048 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:59.758248 14120 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:59.798950  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 50503.4us
I20260709 13:01:59.870836  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:01:59.837420  8744 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:59.744081 14032 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.668242 14022 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.762758 13722 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:01:59.805982 14145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.013s	sys 0.041s
I20260709 13:01:59.845605  8748 raft_consensus.cc:2410] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:59.834930 14051 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.904753  8815 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:59.806677 14035 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.912425 14035 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:59.906762 13958 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:01:59.925377 14164 consensus_queue.cc:1175] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b60591217ee24378bb66a86d6d0dda52" responder_term: 18 status { last_received { term: 14 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [15.11-17.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:59.799332 14151 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.927534 14076 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:01:59.862285 14111 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:01:59.929538 14111 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.930061 13765 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.867650 14053 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:01:59.867730 13978 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:01:59.933871 13978 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:01:59.934397 14175 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:01:59.934816 13978 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.935667 14110 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:59.936072 14110 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:01:59.936272 14170 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 13:01:59.823216 14173 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:01:59.903481 13869 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:01:59.938059 14177 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:01:59.938547 13869 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.939075 13869 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:01:59.942387  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.713860 14067 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.714008 14167 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:59.827400  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.828122  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.828331  8741 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:59.852185  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.738095 14028 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
W20260709 13:01:59.831493  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.828685  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:01:59.949667  8742 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:59.945473  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.951243  8894 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:01:59.944541  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.952164  8893 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:01:59.737452 14168 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:01:59.858297  8645 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:59.824450 14063 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:01:59.859371  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:01:59.762728 14047 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
W20260709 13:01:59.865526  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114740us
I20260709 13:01:59.987787  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:01:59.805924 13419 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:59.846238 13368 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:59.818207 14075 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:59.820487 14040 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:59.864049 13813 raft_consensus.cc:1064] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
W20260709 13:01:59.862852 14171 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:01:59.846671 14172 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:01:59.993930  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 185575us
I20260709 13:01:59.822307 14054 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:59.992718  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:01:59.991494  8890 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:01:59.985379  8645 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 13, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 13 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:00.005986  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 146703us
I20260709 13:01:59.907524 14051 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:00.056152  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68513.3us
W20260709 13:02:00.008935  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58094.6us
W20260709 13:02:00.056612  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 99861.4us
I20260709 13:01:59.908403 13722 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:01:59.977770 14167 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:01:59.888365 14181 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 13:02:00.100270  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:01:59.977629 13958 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:01:59.981554  8746 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:00.208241  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:00.214812  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:00.101123  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.101218  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 346256us
I20260709 13:01:59.980154 13869 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:01:59.936892 14110 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.102463  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:00.110607  8742 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:00.059031 14069 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:00.060474  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 315343us
W20260709 13:02:00.235630  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 364926us
I20260709 13:02:00.004515 14071 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.991045 14021 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.004757 13917 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.182354  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:00.186997  8743 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:02:00.197880  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.227846  8744 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:00.102367 14040 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:00.114108  8749 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:02:00.190217 14073 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:00.182782 14148 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:00.231256  9441 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
W20260709 13:02:00.234077  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 131729us
I20260709 13:02:00.237389  8645 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 12, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 12 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:00.103857 14054 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:00.228119  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:00.179320 14067 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.247232  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:00.271845  8893 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:00.238781 13368 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:00.242267 14169 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.351s	user 0.001s	sys 0.000s
I20260709 13:02:00.229606  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:00.250478  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:01:59.990871 13765 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.238s
I20260709 13:02:00.284533  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:00.285393  8742 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:00.253925  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.286500  8742 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:00.191344 14048 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:00.203156 14167 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:00.247892  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60028.4us
I20260709 13:02:00.290176  8890 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:00.239434 14047 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:02:00.272315  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49007.1us
I20260709 13:02:00.299296  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:00.271410 13869 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:02:00.275565  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:00.300455  8744 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:00.300705 14178 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.504s	user 0.001s	sys 0.000s
I20260709 13:02:00.301791  8744 raft_consensus.cc:1217] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.10->[10.11-14.13]   Dedup: 10.11->[11.12-14.13]
I20260709 13:02:00.302381  8744 pending_rounds.cc:85] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 11
I20260709 13:02:00.302672  8744 pending_rounds.cc:107] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:02:00.302901  8744 raft_consensus.cc:2889] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:00.303982 14174 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.437s	user 0.000s	sys 0.001s
I20260709 13:02:00.276081 13997 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:00.278578 14182 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:00.310640  8893 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:00.282393  8896 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:00.243657 14151 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:00.283442  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2380) took 1275 ms. Trace:
I20260709 13:02:00.229782  8746 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:00.260845  9441 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
W20260709 13:02:00.266969  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79505.9us
I20260709 13:02:00.282823 14069 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.256589 14110 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:00.301874 14197 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.274565 14054 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.452265 14205 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.139s
I20260709 13:02:00.454998  8893 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:00.455576 13961 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:00.311429  8892 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:02:00.312142  8898 rpcz_store.cc:276] 0709 13:01:59.008318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.008471 (+   153us) service_pool.cc:224] Handling call
0709 13:01:59.008797 (+   326us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:59.081879 (+ 73082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.081910 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:59.087155 (+  5245us) log.cc:844] Serialized 78 byte log entry
0709 13:01:59.482151 (+394996us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.790440 (+308289us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.792409 (+  1969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.792506 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.283186 (+490680us) raft_consensus.cc:1685] finished
0709 13:02:00.283210 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.283378 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81743,"prepare.queue_time_us":469514,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":103781,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":24902,"thread_start_us":635389,"threads_started":2,"wal-append.queue_time_us":487599}
W20260709 13:02:00.459489  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 160291us
I20260709 13:02:00.460554  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:00.246485 14071 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.248838 14021 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.248703 13917 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.295122  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:00.321805  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:00.323027  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:00.331980  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.442330 14028 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:00.443077 14048 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.443542 14183 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
I20260709 13:02:00.444034 14069 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.299255 14053 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.447742  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:00.448571 13368 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.436981 14151 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.450245 14206 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:00.462081  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:00.458952  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 137037us
W20260709 13:02:00.468525  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 173510us
I20260709 13:02:00.469035  9449 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 12: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:00.470476 14181 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.265861 14148 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.464016 13898 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.436615 14045 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.839s	user 0.011s	sys 0.000s
I20260709 13:02:00.467303 13933 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:00.462077 14166 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.822s	user 0.000s	sys 0.004s
I20260709 13:02:00.465181 14197 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:00.479806 14115 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.002s
I20260709 13:02:00.313879 14073 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.309551 14182 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:00.481426  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 158524us
I20260709 13:02:00.487517  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:00.487893  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.488680  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.492512  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:00.503151  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.466145 14205 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:00.477355 14199 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:00.512989  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.540223  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:00.474865  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.506330 13869 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.449790 14040 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.518904 14069 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.519982 14200 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:00.516474 13898 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:00.474426 14123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.010s
I20260709 13:02:00.479420 13839 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:00.476538 14071 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.322618 13997 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.476199 14021 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:00.479333 13917 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.300400 13958 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.487469 14036 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.530704  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:00.532037  9444 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:00.484822 14145 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:00.509315  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.534121  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:00.653987  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 120006us
I20260709 13:02:00.611685 14200 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:00.535876  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:00.480792 14207 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:00.505473 14183 consensus_queue.cc:1243] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:00.533239 14149 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.543839  8819 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:00.553027 14204 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.170s
I20260709 13:02:00.546977  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:00.553753  8748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 911) took 1628 ms (client timeout 2133 ms). Trace:
I20260709 13:02:00.555934  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:00.671142  8748 rpcz_store.cc:269] 0709 13:01:58.925146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.925269 (+   123us) service_pool.cc:224] Handling call
0709 13:02:00.553461 (+1628192us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400160dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:00.553670 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099856640}
I20260709 13:02:00.485589 14028 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.561625  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:00.590063  9448 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:00.590055 13978 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:00.495532 14053 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:00.606107  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 144151us
I20260709 13:02:00.515805 14045 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.508249 14181 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.530834 14115 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:00.602491 14123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.005s	sys 0.010s
I20260709 13:02:00.530416 13933 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.624027  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:00.534391 14073 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.538740 14063 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 13:02:00.618027 14074 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.157s	user 0.012s	sys 0.000s
I20260709 13:02:00.617746 13839 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:00.479691 13722 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.657225  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2373) took 1735 ms. Trace:
I20260709 13:02:00.555941 13869 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:00.693661 14213 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 13:02:00.659822  9444 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:00.659523  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:00.660876  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.712186  8895 rpcz_store.cc:276] 0709 13:01:58.922162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.922292 (+   130us) service_pool.cc:224] Handling call
0709 13:01:58.922597 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:58.922943 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.922971 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:58.932365 (+  9394us) log.cc:844] Serialized 52 byte log entry
0709 13:01:59.315715 (+383350us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:59.315772 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.315853 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.452685 (+1136832us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:00.650809 (+198124us) raft_consensus.cc:1685] finished
0709 13:02:00.650830 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.657166 (+  6336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80616,"raft.queue_time_us":7158,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":50967,"spinlock_wait_cycles":17947008,"thread_start_us":279769,"threads_started":1,"wal-append.queue_time_us":574020}
I20260709 13:02:00.716878  8788 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:00.544756  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:00.626853 14035 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.001s
I20260709 13:02:00.603590 14051 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.614678 14209 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:00.542306 14199 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:00.660531 14075 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.003s
I20260709 13:02:00.686395  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:00.654948 14145 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:00.689143  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:00.693977  8890 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:02:00.722693 14219 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.669818 14218 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.689793 14149 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.698665  9443 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:00.693049 13978 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:00.740855 14217 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:02:00.741869 13978 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.707048  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:00.742702  8819 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:02:00.713073  8824 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:00.658069  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:00.670672 13917 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:02:00.717861  8821 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:02:00.720811  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61378.3us
I20260709 13:02:00.673996 14069 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:02:00.681485 14071 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:00.661168 14167 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.711287 14074 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:00.770576  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.729074  8749 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:02:00.622119 14182 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.779789  8749 pending_rounds.cc:85] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:00.780225  8749 pending_rounds.cc:107] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:02:00.780603  8749 raft_consensus.cc:2889] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.733656  9447 raft_consensus.cc:2368] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16.
W20260709 13:02:00.748173  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72420.7us
W20260709 13:02:00.761487  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75233.7us
I20260709 13:02:00.724400 14063 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.726558 14008 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:00.852486  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.733022 14218 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:00.733250 14021 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.732476 14151 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.733044 14148 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.857079  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.858453  8749 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:00.860031  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.764242 14183 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.860849  8823 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:00.861117  8818 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:00.730134 14075 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.859756 14218 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.762389 14149 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.859424 14197 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.877234  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:00.879784  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:00.856252 14063 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:00.882148 14111 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.952s	user 0.010s	sys 0.001s
I20260709 13:02:00.870528 14145 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.886669 13869 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:00.888443 14075 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.871065 14054 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.729755 14200 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.911504  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:00.873504 13898 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.873502 13997 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.921244  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:00.926257  8742 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:00.928089 14221 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:00.942778 14063 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.944202  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:00.918599 14045 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:00.949456  8823 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:00.949479  8749 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:00.921721 13815 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.866041 13839 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:00.922776 14229 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:00.965409  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:00.966387  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.973765  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:00.961880 14008 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:00.914491 14111 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:00.980384  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:00.963709 14115 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:00.961601 14028 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.981954  8749 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:00.968524 13933 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:00.985565 14223 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:02:00.986624 14149 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.985502 14051 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.987789 14199 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.994660  8823 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:02:00.995256  8823 pending_rounds.cc:85] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 7
I20260709 13:02:00.995546  8823 pending_rounds.cc:107] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:00.995852  8823 raft_consensus.cc:2889] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.997249  8823 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:00.998462  8823 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:00.998826  8823 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:01.000716  8823 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.001405 14236 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.999374 14008 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.002338  8823 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.965895 13815 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.005777  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:01.008833  9442 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:00.984212 14181 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:00.984608 14054 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:00.972379 14229 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:01.019757  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.030367  8823 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:02:01.031337  8823 pending_rounds.cc:85] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 6
I20260709 13:02:01.031659  8823 pending_rounds.cc:107] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:01.032009  8823 raft_consensus.cc:2889] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.033708  8823 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:01.033802 14021 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.002121 14236 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:01.044075 14206 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:02:01.044180 14120 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:01.044791 14120 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:01.049557 14111 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:01.041085 14228 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:02:01.066437 14236 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.103456  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.083829 14073 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:01.064096 14071 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.084540  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.088532 14212 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.088914  8820 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:01.097656  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.098750  8891 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:02:01.108410 14067 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.814s	user 0.012s	sys 0.000s
I20260709 13:02:01.111405 14067 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.111920  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.113782  8819 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:01.114542  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.101728  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:01.117115  8892 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.101277  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.104650  8896 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:01.081732 14120 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.109086  8749 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:01.113893  8897 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:01.094252  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.128223 14120 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:01.138353  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.139029  9447 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:01.139806  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.116178 14212 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.156720  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.160480  8892 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:01.160605  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.162482  9447 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:01.163622  9447 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:01.160714  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:01.166257  9447 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:01.119158 14071 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.182472 14053 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:01.183127 14212 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.145143 14167 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:01.187096 14242 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:02:01.190548  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.191293  8823 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:01.191905 14071 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.193379 14229 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:01.194527  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 63356.7us
I20260709 13:02:01.202515 14053 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:01.203358 14196 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.684s	user 0.002s	sys 0.000s
I20260709 13:02:01.203701 14053 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.203720  8892 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:01.207231  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:01.209235 14229 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:01.209801 14229 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:01.210677 14229 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.214423  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.214332  8898 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:01.212661 14159 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 13:02:01.220919  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 908117us
I20260709 13:02:01.222235 14053 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.222062 13839 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:01.223171  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:01.223359 14053 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:01.224263 13839 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:01.224828 13839 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:01.225180 14218 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.225845 13839 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.226413 14230 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:01.226869 14230 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:01.227710 14230 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.229010 14230 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.224279 14053 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:01.232545 14053 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:01.227386 14218 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.234256 14053 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.236601 14053 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.240051 14195 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.099s	user 0.000s	sys 0.004s
W20260709 13:02:01.241080  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.243851  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.246780  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.248275  8891 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.252359  8896 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:01.252792  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.251338  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.253482  8892 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:01.244920 13578 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.244815 14045 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:01.256876 14045 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.259559 14045 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.260625  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.266865 14116 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.267349 14116 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:01.264509  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.269518  8864 connection.cc:256] RPC call timeout handler was delayed by -1.0868075440000666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 13:02:01.264920 13578 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.273092 13578 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.280462 13912 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:01.280954 13912 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:01.282099 13912 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.283522 14148 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.284104 14148 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:01.285082 14148 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.286271 14148 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:01.286511  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 154926us
I20260709 13:02:01.268335 14116 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.289992  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:01.292536  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.293545  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.293650  8891 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:01.283691 13912 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.297425  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:01.307039  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43883.5us
I20260709 13:02:01.307272  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:01.308880  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.309311  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:01.313155  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:01.314112 14186 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.314570  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.298949 14212 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:01.317185  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.319422  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.298652 14054 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.321242  8897 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:01.313649 13839 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.327831  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.316414 14116 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.329895  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:01.332086 14119 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:01.332387 14054 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:01.325533 14073 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.361938  8897 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:01.333634 13419 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:01.333823 14062 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:01.334467 14176 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.362473 14073 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.335489 14220 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:01.341430  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:01.342463  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2328) took 1222 ms. Trace:
I20260709 13:02:01.343614 14214 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:01.347275  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.375396  8746 rpcz_store.cc:276] 0709 13:02:00.120110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.314796 (+194686us) service_pool.cc:224] Handling call
0709 13:02:00.315152 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.529304 (+214152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.529341 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.535809 (+  6468us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.686250 (+150441us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:00.686286 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.341988 (+655702us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.342067 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.342143 (+    76us) raft_consensus.cc:1685] finished
0709 13:02:01.342161 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.342402 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491918592,"thread_start_us":94494,"threads_started":1,"wal-append.queue_time_us":137842}
I20260709 13:02:01.348815  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.349222  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:01.377228  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.377512  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.378806 14021 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.378929  8894 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:01.379132  8823 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.351063 14076 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:01.352612  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:01.353518  9441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 1058) took 1769 ms (client timeout 1547 ms). Trace:
W20260709 13:02:01.381459  9441 rpcz_store.cc:269] 0709 13:01:59.584488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.584682 (+   194us) service_pool.cc:224] Handling call
0709 13:02:01.353303 (+1768621us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002eb8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:01.353426 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256841600}
I20260709 13:02:01.315598 13368 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.355213  9444 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:01.310333 14020 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:02:01.320530 13827 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:01.323843 14115 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.392436  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 916) took 1737 ms. Trace:
I20260709 13:02:01.363821  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2327) took 1244 ms. Trace:
I20260709 13:02:01.366374 13733 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:01.365767  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:01.334923  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45011us
W20260709 13:02:01.364507 14250 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:01.354945 13839 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:01.384691  8824 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:01.363216 14167 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:02:01.393440  8744 rpcz_store.cc:276] 0709 13:02:00.119767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.301171 (+181404us) service_pool.cc:224] Handling call
0709 13:02:00.301581 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:00.305936 (+  4355us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:00.440291 (+134355us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.204885 (+764594us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b18002cac40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:02:01.238811 (+ 33926us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:01.241193 (+  2382us) log.cc:844] Serialized 52 byte log entry
0709 13:02:01.241429 (+   236us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:01.241475 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.241574 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.355590 (+114016us) raft_consensus.cc:1685] finished
0709 13:02:01.363529 (+  7939us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.363750 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83,"prepare.queue_time_us":134121,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":9094,"raft.queue_time_us":4943,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":4747,"spinlock_wait_cycles":52898304,"thread_start_us":33563,"threads_started":1,"wal-append.queue_time_us":857740}
I20260709 13:02:01.389133  8897 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:02:01.391047 14021 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:01.392297 14071 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:01.392976  8740 rpcz_store.cc:276] 0709 13:01:59.655325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.664584 (+  9259us) service_pool.cc:224] Handling call
0709 13:01:59.713218 (+ 48634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.713467 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:59.855972 (+142505us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.391749 (+1535777us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b1800345640. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:02:01.391994 (+   245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.392057 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:01.392115 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.392226 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.392367 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193728,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2231,"spinlock_wait_cycles":245571328,"thread_start_us":257524,"threads_started":2,"wal-append.queue_time_us":439297}
I20260709 13:02:01.404561  8743 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:01.382961 14220 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.406201  8897 pending_rounds.cc:85] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 13
I20260709 13:02:01.412972  8897 pending_rounds.cc:107] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:02:01.410234  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.382606 14176 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.413461  8897 raft_consensus.cc:2889] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.403373 14036 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.412221 13419 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.424723 14036 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.411903 13709 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.425166 13419 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.408416 14214 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:01.425755 13709 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.426296 14214 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:01.405973 13839 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.434183  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.434532  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.406620 14167 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.413055 14220 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:01.438504  8897 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:01.425218 14036 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.449095  9442 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.428333 14230 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.414485 14062 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:01.460868 14062 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:01.461361 14062 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:01.462275 14062 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.463567 14062 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.466027  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.466472  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.470670 13709 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.472515 13419 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:01.477876 14063 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.478699 14063 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.482174 14063 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.484166 13912 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.484781 13912 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.485611 14234 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:01.486152 14234 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.487002 13912 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.489948  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:01.490303 14234 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.490489  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.408766 14071 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.456792  8823 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:01.437227 14167 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:01.515960 14253 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:02:01.517179 13709 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.517786 13709 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:01.519598 14252 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:02:01.519995 14182 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.520262 13709 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.500765 14047 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.526503  8892 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:02:01.527149 14033 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:01.525164  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.535195  9442 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:01.528657 13578 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.495292 14230 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.550371 14073 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.533406 14033 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:01.567006  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2593) took 1610 ms. Trace:
I20260709 13:02:01.567556  8890 rpcz_store.cc:276] 0709 13:01:59.956239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.304980 (+348741us) service_pool.cc:224] Handling call
0709 13:02:00.443233 (+138253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.727055 (+283822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.566662 (+839607us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540015d954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.566917 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575983488}
I20260709 13:02:01.573212  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2087) took 1080 ms. Trace:
I20260709 13:02:01.573611  8819 rpcz_store.cc:276] 0709 13:02:00.492391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.880887 (+388496us) service_pool.cc:224] Handling call
0709 13:02:00.881349 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.114471 (+233122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.572902 (+458431us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54000825d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.573143 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533999360}
I20260709 13:02:01.510356 14214 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.589618  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:01.591034  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:01.593015  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.572525 14151 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.588965  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.593920  8822 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:01.604023  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.604628  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.606174  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.607337  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.607781 13813 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.608683  8823 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:01.609413 14234 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:01.609900 14234 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.611913  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1068) took 1289 ms. Trace:
I20260709 13:02:01.611999 14063 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.612543 14063 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.612362  9446 rpcz_store.cc:276] 0709 13:02:00.322662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.322841 (+   179us) service_pool.cc:224] Handling call
0709 13:02:00.323235 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.517515 (+1194280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.517544 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.517583 (+    39us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:01.517623 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.611701 (+ 94078us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540065f4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.611747 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.611856 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2746488064}
I20260709 13:02:01.613299 14062 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.613783 14054 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:01.616168  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 925) took 1085 ms. Trace:
I20260709 13:02:01.553937 14167 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.626219  9440 rpcz_store.cc:276] 0709 13:02:00.530845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.531113 (+   268us) service_pool.cc:224] Handling call
0709 13:02:00.531418 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.531720 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.531745 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.532156 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.426844 (+894688us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18003253c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:01.498885 (+ 72041us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:01.498926 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.499026 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.615859 (+116833us) raft_consensus.cc:1685] finished
0709 13:02:01.615883 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.616112 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11257,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9696,"spinlock_wait_cycles":13178112,"thread_start_us":71835,"threads_started":1,"wal-append.queue_time_us":1072239}
I20260709 13:02:01.624320  8822 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:01.624730  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.617048  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:01.640125  8819 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
W20260709 13:02:01.629504  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.632280  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 899) took 1157 ms. Trace:
I20260709 13:02:01.646297  8818 rpcz_store.cc:276] 0709 13:02:00.483807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.490382 (+  6575us) service_pool.cc:224] Handling call
0709 13:02:00.495820 (+  5438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.863785 (+367965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.631977 (+768192us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54003e30d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.632207 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839788928}
I20260709 13:02:01.639415  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:01.656396  8740 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:02:01.618899 14053 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:01.625267 14214 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.674392  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.625576 14168 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.675534  8818 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:01.676693  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.660660  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.678999  8823 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:01.619246 14176 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.656884  8740 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 7
I20260709 13:02:01.685237  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.685691  8740 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:02:01.686172  8740 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.625926 14054 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.682763  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.690321  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:01.658252 14122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.013s	user 0.020s	sys 0.046s
I20260709 13:02:01.691550  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.697638  8819 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:01.617410 14040 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:02:01.695117  9440 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:01.703256  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.659606 14167 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.704427  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.653781 14230 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.721621  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:01.722350  8740 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.722640 14063 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.723291 14176 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.724869 14230 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.725463 14229 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:01.619167 14151 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.732139 14151 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:01.699182 14054 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:01.735556 14234 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:01.737409  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.738435 14194 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:01.738855 14194 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:01.739722 14194 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.740713 14194 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:01.693327 14053 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:01.742214 14053 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.706421  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 999) took 2596 ms. Trace:
I20260709 13:02:01.745456  9443 rpcz_store.cc:276] 0709 13:01:59.109926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.110108 (+   182us) service_pool.cc:224] Handling call
0709 13:01:59.110425 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.732438 (+1622013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.706064 (+973626us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400197654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:01.706344 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3652408192}
I20260709 13:02:01.746776  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.748116  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.748875  8819 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:01.710291  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:01.716687 14211 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.193s	user 0.005s	sys 0.000s
I20260709 13:02:01.750952  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2673) took 1953 ms. Trace:
I20260709 13:02:01.751451  8816 rpcz_store.cc:276] 0709 13:01:59.797981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.798170 (+   189us) service_pool.cc:224] Handling call
0709 13:01:59.798617 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.060000 (+261383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.060044 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.117352 (+ 57308us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.298264 (+180912us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.298316 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.299737 (+  1421us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.623940 (+1324203us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:01.750540 (+126600us) raft_consensus.cc:1685] finished
0709 13:02:01.750573 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.750872 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60266,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":22968,"spinlock_wait_cycles":19598592,"thread_start_us":299582,"threads_started":2,"wal-append.queue_time_us":278142}
I20260709 13:02:01.753708  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.754312  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.627538 14033 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.768615 14036 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.758646  8745 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:01.779019  8745 pending_rounds.cc:85] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 8
I20260709 13:02:01.779352  8745 pending_rounds.cc:107] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:02:01.779633  8745 raft_consensus.cc:2889] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.759174  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:01.766304  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.767751  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.782081  8815 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:01.783850  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.784610  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.707973 13815 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:01.701561 13709 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:01.772856  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.788448  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:01.788842  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.774035  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:01.774374 14198 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.306s	user 0.005s	sys 0.001s
I20260709 13:02:01.677569 13839 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:01.696946 14122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.060s	user 0.021s	sys 0.046s
I20260709 13:02:01.716590 13912 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.778229  8816 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:01.757658  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:01.785964  8746 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:01.769873  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.780825 14235 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.791057  8740 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:01.816483  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.777423  8822 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:01.798207  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.815167  8819 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:01.820752  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:01.782063 14207 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:02:01.827040  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.832880  8746 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:01.834075  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.839664  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:01.827489  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.801119 14076 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:01.830080  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.789654 14033 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:01.849867  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.850600  8815 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:01.813609 14122 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:01.813868 14148 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:01.826198 14063 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.832273 14151 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:01.843613 14032 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:01.847925 14076 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.837710 14234 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:02:01.872545  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:01.872591  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45179.2us
I20260709 13:02:01.858817 14151 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.843380 14220 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.832305 14235 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.876796  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:01.863891 14032 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:01.881806  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:01.867225 14076 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.888921  8815 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:01.889842  8815 raft_consensus.cc:606] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.857822 14054 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.876569 14230 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:01.859247 14148 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.878810 14151 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:01.882826 14063 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.912948  8815 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:01.890861 14176 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:01.918757  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.891139 13912 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.923529  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.893154 13810 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.929857 14262 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:01.933935  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.934263  8823 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:01.934944  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.935667  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.936347  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.936950  8895 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:01.915652 14063 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.938264  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:01.939117  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.939309  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.940867  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:01.941834  8749 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:01.942514  8892 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.943128  8895 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:01.944748  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.947918  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 905) took 1200 ms. Trace:
I20260709 13:02:01.948510  8821 rpcz_store.cc:276] 0709 13:02:00.747349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.762265 (+ 14916us) service_pool.cc:224] Handling call
0709 13:02:01.938958 (+1176693us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b580013fe98. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33aaa22 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4950a0 00007f90fb494984 00007f90f4a0f6d4 00007f90f4a0f662
0709 13:02:01.947835 (+  8877us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3307392}
W20260709 13:02:01.948114  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.949980  8824 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:01.921975 14176 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:01.899560 14032 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.946396 14063 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.956331  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:01.957649  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:01.979566  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.937549 13810 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.986102  9444 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 15 LEARNER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:01.983690  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:01.894992 14122 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:01.996100  8821 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:01.995878 14094 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:02.026192 14094 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:01.983285 14207 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.028591 14094 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.028509 14207 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.030068 14067 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:01.995919 14073 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.072136  8749 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:02.072520  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:01.953747 14036 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.033362  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:01.987609 14214 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:01.989889 13839 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:01.955106 14176 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:02.036177  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.080470  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.027945 14235 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.044350  8892 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:02.085783  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.086410  8892 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:02.086935 13810 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.087270  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:02.045567  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.071844 14032 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:02.088734  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:01.931847 13912 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.032454  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.070053 14038 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.089951  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:02.090590 14038 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.093961 14148 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.094509 14038 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.092339 13912 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.094873 14123 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:02.096354  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:02.097527  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.076774  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:02.073190  8821 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:02.098796  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:02.099205 14266 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:02:02.099324  8821 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:02.099850  8749 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:02.100148  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.100293  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.078328  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.101202  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.101881  8894 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:02.102169  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.045224 14268 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.102926 14268 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.047356 14206 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.105593  8893 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:02.105680  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 898) took 1636 ms. Trace:
I20260709 13:02:02.106129  8817 rpcz_store.cc:276] 0709 13:02:00.469339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.469468 (+   129us) service_pool.cc:224] Handling call
0709 13:02:00.480040 (+ 10572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.746340 (+266300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.746371 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.746938 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.957227 (+210289us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:00.957317 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.105310 (+1147993us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:02.105363 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.105414 (+    51us) raft_consensus.cc:1685] finished
0709 13:02:02.105427 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.105623 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611861248,"thread_start_us":99264,"threads_started":1,"wal-append.queue_time_us":231323,"wal-append.run_cpu_time_us":2596,"wal-append.run_wall_time_us":1032688}
I20260709 13:02:02.037804 14173 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:02.108000 14173 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.082875  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.080965 14073 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.110077  8897 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:02.070204 14067 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:02.089183 14220 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:02.111449 14073 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.111682 14067 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.112877 14067 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.048836 14062 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.105206 14206 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.118428  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.125432  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.126233  8894 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:02.118031  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.115142  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.123571 14115 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:02.112969 14173 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.128028  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.130745  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.131880  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.132474  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:02.138500  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.082091 14214 consensus_queue.cc:1243] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:02.138476  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:02.141266  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.142376  8821 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:02.123809 13839 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:02.113857 14268 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.095096 14054 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:02.139240 14036 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.125694 14040 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:02.115258 14207 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.122525 14111 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:02.154191 13709 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:02.140820 14038 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:02.142334 14115 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:02.139649 14062 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.127490 14094 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:02:02.139871 13815 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:02.151718  8818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:02.168761  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.183831  8818 raft_consensus.cc:606] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.142436 14182 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.147213 14173 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:02.149471 13839 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.124327 13813 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:02.180781  8746 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:02:02.151432 14168 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:02.157197 14071 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.215870 13815 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:02.184635  8818 raft_consensus.cc:614] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:02.169741 13912 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:02.181097 14040 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:02.228792  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.202597 14206 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.203867  8743 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:02.205703 14062 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.196904 14076 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.188650 14173 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.207196 13709 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:02.207661 13810 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:02.210562  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.232203 14063 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.170032 14038 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.217192 14168 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.208405 14094 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:02:02.220955  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.216096 13839 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:02.219614 14071 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:02.213368 14182 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.233263  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.240743  8821 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:02:02.239810  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 203131us
I20260709 13:02:02.213773  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:02.226891 14230 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:02.252811  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 217354us
I20260709 13:02:02.250490 14234 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.256193 14168 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.233609 14022 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:02.255268  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:02.262343  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 928) took 1788 ms. Trace:
I20260709 13:02:02.263659  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2065) took 2429 ms. Trace:
W20260709 13:02:02.269757  8820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 902) took 1781 ms (client timeout 1722 ms). Trace:
I20260709 13:02:02.296175  8742 rpcz_store.cc:276] 0709 13:02:00.474270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.724373 (+250103us) service_pool.cc:224] Handling call
0709 13:02:00.730699 (+  6326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.926623 (+195924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.262009 (+1335386us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400463cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:02.262262 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442426624}
I20260709 13:02:02.243127 14173 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.272979  8749 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:02:02.275477  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 153043us
I20260709 13:02:02.296761  8745 rpcz_store.cc:276] 0709 13:01:59.834507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.838156 (+  3649us) service_pool.cc:224] Handling call
0709 13:01:59.838517 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.780090 (+1941573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.263311 (+483221us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54004646d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:02.263525 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4415609088}
I20260709 13:02:02.299790  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.300833  8745 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:02.242244 13709 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.279744  8817 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:02.304816  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.256455  8740 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:02.251762 14038 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:02.287357  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:02.309973  8896 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.310141  8893 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:02.311290  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.312542  8742 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:02.313848  8742 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:02:02.313903  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 107218us
I20260709 13:02:02.291409 14230 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:02.291932  8890 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:02.292109  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.256503 14032 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.251837 14073 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.270967  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:02.309638 14194 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.005s
I20260709 13:02:02.328032  8889 raft_consensus.cc:2410] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:02.301056  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.304064  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.248080 14176 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:02.307531  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.333393  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2390) took 1086 ms. Trace:
I20260709 13:02:02.289930  8862 leader_election.cc:400] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16.
I20260709 13:02:02.333983  8862 leader_election.cc:403] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:02.315009  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:02.322431  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.296919  8820 rpcz_store.cc:269] 0709 13:02:00.488323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.488449 (+   126us) service_pool.cc:224] Handling call
0709 13:02:02.269500 (+1781051us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:02.269653 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284202752}
I20260709 13:02:02.324188  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.335662  9444 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:02.324365  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.325861  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.326398  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.326951  9446 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:02.324522  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.308667 14094 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:02.342746  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.338752  9445 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:02.344406  8744 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:02.309438 13709 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:02.313041 14235 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:02.276180 14207 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:02:02.333766  8889 rpcz_store.cc:276] 0709 13:02:01.246724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.270677 (+1023953us) service_pool.cc:224] Handling call
0709 13:02:02.333330 (+ 62653us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:02.296583 14234 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:02.374697 14273 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:02.384616 14076 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.310238 13815 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.385568 14076 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:02.386155 14076 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:02.386683 14076 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.306423 14022 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:02.401044  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.338783  8896 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
W20260709 13:02:02.341704  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 51991.9us
I20260709 13:02:02.330844 14073 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.345805 14047 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:02.346766 14271 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:02:02.347110 14063 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.349323 14094 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.349450 14286 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:02.291841 14115 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.329679 14032 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.355755 14284 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.360193 14182 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.297730 14054 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:02:02.365108  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.366281  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.458519  9443 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:02.367292  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.459632  9440 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:02.368295  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.310186 14040 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.418372  8817 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:02.421511  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.421669  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.422582  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:02.429843 14151 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.001s
I20260709 13:02:02.414654 13419 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16.
I20260709 13:02:02.439715  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:02.429911 14038 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.445051 14094 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.451043  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:02.464706  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 99690.3us
W20260709 13:02:02.467674  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45223us
I20260709 13:02:02.437515 14047 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.483043  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.484001  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:02.485415  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 170543us
W20260709 13:02:02.487350  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 47774.3us
I20260709 13:02:02.469770 14182 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:02.491600  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.473201 14194 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.456501 13813 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:02:02.490939  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:02.494506  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 170231us
I20260709 13:02:02.498039  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.501437 14062 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:02.501533  8787 leader_election.cc:336] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.633s (SENT)
I20260709 13:02:02.501905 14176 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.502197 14032 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.503270  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.503526 14073 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.503785 14268 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.504385 14206 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:02.502009 13733 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.474575 14289 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:02.511981  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.503065 14038 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.522926  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.496002 14051 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:02.475265 14151 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.497241 14182 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.492663 14282 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.492671 14048 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:02:02.530072 14111 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:02.537835  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.542886  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:02.566151  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.521727 14294 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:02.576473 14151 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.533034 14032 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.588469  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45724.8us
I20260709 13:02:02.589622  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.590854 14022 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:02.584697 14294 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.591831 14294 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:02.549139 14182 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:02.597602 14292 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:02:02.600889 14287 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:02:02.602969  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.603821  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:02.549576 14063 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:02.557693 14038 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:02.586756  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.610599  8713 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.583082 14054 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.536144 14230 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.540234 13733 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.644269  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.583866 14277 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:02.536902 14268 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:02.712497  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68369.9us
I20260709 13:02:02.714160  8896 raft_consensus.cc:1217] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[14.9-16.10]   Dedup: 14.9->[16.10-16.10]
W20260709 13:02:02.715994  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 415058us
I20260709 13:02:02.721747  8896 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: [14.9-16.10]
I20260709 13:02:02.720743  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:02.722858  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.608603 14176 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:02.725613  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 209030us
I20260709 13:02:02.612591 14151 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:02.708115 14071 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.729938  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:02.621937 14234 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.730756  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:02.730091  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:02.725178  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 195411us
I20260709 13:02:02.736507 14302 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.766105 14302 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.768538 14302 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:02.732498  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 234564us
I20260709 13:02:02.773525  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:02.774147  9446 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:02.732460 14296 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.776590  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:02.778282 14167 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:02.778693 14167 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:02.779456 14167 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.608889 14032 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.600821 14294 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.790566  8824 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:02.729126 14277 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:02.780390 14167 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:02.782086  8817 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:02.797627  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.784289  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.785250  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:02.785555  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.786381  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:02.789477  8745 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:02.707511 14054 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.739318 14063 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:02.803397  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:02.798478  8816 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:02.800737  8819 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:02.783592  8820 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:02.808187  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:02.808471  8818 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:02.809149  8819 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:02:02.815013  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 85002.4us
I20260709 13:02:02.818764  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.759193 14176 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.806012  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.824469  8820 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:02.824853  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.827160  8820 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:02.833971  8822 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:02.825704  8741 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:02.839828  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:02.840776  9446 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:02.840652  8824 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:02.840652  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:02.843755  8741 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:02:02.842214  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 128071us
I20260709 13:02:02.849166  8744 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:02.803718 14063 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.803855 14234 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.852257  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:02.804046 14296 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:02.803480 14214 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:02.848259 13813 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:02:02.856055 14145 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:02.865383  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.873324 13813 consensus_queue.cc:1243] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:02:02.878373 14301 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:02:02.863540  8788 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:02.863831  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:02.892366  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 165321us
I20260709 13:02:02.893203  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:02.894063  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.894986  8744 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:02.858305 14063 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.852037 14047 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:02.923563 14303 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:02:02.853906 14308 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.928391 14307 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:02.932158  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68409.3us
I20260709 13:02:02.929993  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.924829 14206 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.942927 14206 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.933574 13815 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:02:02.928157 14306 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:02.938856  8744 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:02.946565 14094 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.947089  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:02.948557  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.949540 14206 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:02.950065  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:02.950644  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:02.951283 14032 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.951406  8744 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:02.955027 13839 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.002s
I20260709 13:02:02.955529 13839 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.933782 14300 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:02.957499 14054 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:02.958388 13733 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:02.960783 14032 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:02.950680  9443 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.964601  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:02.904305 14038 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:02.942193 14307 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.965911 14038 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.966090 14307 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:02.966415 14038 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.967519 14306 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:02.891997 14207 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:02.969049 14071 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:02.969648 14071 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.969447 14207 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.970424  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2080) took 1297 ms. Trace:
I20260709 13:02:02.970794  8746 rpcz_store.cc:276] 0709 13:02:01.672965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.839340 (+166375us) service_pool.cc:224] Handling call
0709 13:02:02.970259 (+1130919us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540013bad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:02.970369 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782688000}
I20260709 13:02:02.971982  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.972519  8746 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:02:02.974121 13709 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.000s
I20260709 13:02:02.974555 13709 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.975962  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2082) took 1302 ms. Trace:
I20260709 13:02:02.976348  8749 rpcz_store.cc:276] 0709 13:02:01.673538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.100540 (+427002us) service_pool.cc:224] Handling call
0709 13:02:02.100798 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.298711 (+197913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.975705 (+676994us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54001d8b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:02.975905 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395314944}
I20260709 13:02:02.977676  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.978190  8745 pending_rounds.cc:85] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:02.978489  8745 pending_rounds.cc:107] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:02.978730  8745 raft_consensus.cc:2889] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.986400  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.987090  8746 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:02.988144  8744 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:02.995463  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.886011 14214 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:02.866783 14234 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.002077  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:02.950621 14284 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:03.006600  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:02.976550 13709 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:03.017210  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 110977us
I20260709 13:02:02.989605 14094 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.012988  8741 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:02.998140 13733 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.027503  8746 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.028398  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.031615  8749 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:02:03.034679  9440 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:02.988963 14182 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.022538 14062 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:03.037437  8746 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:02:03.026846 14076 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.640s	user 0.017s	sys 0.004s
I20260709 13:02:03.005683  9443 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:03.044585  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.046448  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 80205.6us
I20260709 13:02:03.028764 13709 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.055251  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.040318 14268 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.062139  9443 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:03.050731 14067 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:03.070617 14194 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:02:03.071549  8746 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:03.072046 14071 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.034163 13733 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.063664 14294 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.078953 13839 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.079489 14194 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:03.061053 14206 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.069331 14300 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.093657  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.063194 14207 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:03.094867 14168 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.003s
I20260709 13:02:03.061661 13709 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.088315  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:03.165035  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.046931 14076 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.062951 14054 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:03.062712 13578 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:02:03.207795  8746 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:03.076623  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.066228 14053 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:02:02.981153 14308 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:02.924901 14277 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:03.192716 14298 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:02:03.095309 14311 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:03.253991  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.256496  8748 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:03.093611 14312 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.240365 14309 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:02:03.269232  9443 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:03.263025  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 195954us
I20260709 13:02:03.086804 14067 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:02:03.292181  8746 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:03.259087 14182 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.194872 14206 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.308010  8741 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:02:03.310761  8741 pending_rounds.cc:85] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:03.311025  8741 pending_rounds.cc:107] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:03.311260  8741 raft_consensus.cc:2889] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:03.319840 14038 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.336464  8743 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.337208  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2331) took 1611 ms. Trace:
I20260709 13:02:03.337572  8743 rpcz_store.cc:276] 0709 13:02:01.725376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.235153 (+509777us) service_pool.cc:224] Handling call
0709 13:02:02.235452 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.336279 (+1100827us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b5400360720. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d11c0 00007f90f64d2231 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:02:03.336944 (+   665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.337149 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101170432}
I20260709 13:02:03.338217 14230 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.338968  8743 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:03.320430  9443 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:03.350874 14076 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.355623  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.356657  9443 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.359961 14308 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.361430 14038 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.290764 14319 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:03.362675 14319 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.238425 14168 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.269654 14286 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.395524 14286 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.205040 14300 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.399085 14300 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.401202  9443 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:03.407927  8817 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:03.412215 14182 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.412978 14182 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.374408 14214 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.311038 14145 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:03.418452 14145 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:02:03.377179  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.423373  9444 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:03.319145  8746 pending_rounds.cc:85] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:03.427139  8746 pending_rounds.cc:107] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:02:03.427438  8746 raft_consensus.cc:2889] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:03.355984 14234 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.430145 14326 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.430737 14326 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.433442 14040 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:03.433741 13912 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:03.434144 13912 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.434007 14040 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.437662  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.438269  8822 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:03.438762  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.439486  8817 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:03.290946 14311 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.292357 14312 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.439280  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.379485  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 931) took 2431 ms. Trace:
I20260709 13:02:03.451488  8748 rpcz_store.cc:276] 0709 13:02:00.948615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.948775 (+   160us) service_pool.cc:224] Handling call
0709 13:02:00.949425 (+   650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.269434 (+2320009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.379179 (+109745us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54000b6654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:03.379412 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5297432320}
W20260709 13:02:03.442816  8715 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.130:44185 timed out after 1.780s (SENT) [suppressed 12 similar messages]
I20260709 13:02:03.452945  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.453612  8817 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:03.464990  8822 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:03.456979 14063 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 13:02:03.467408  8820 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:03.499375  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.478711  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.503103  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.503854  8822 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:03.455651 14167 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.000s
I20260709 13:02:03.450881 14067 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.536371  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.458794 14319 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:03.470579 14173 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 1.155s	user 0.008s	sys 0.005s
I20260709 13:02:03.463126 14145 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.440600 14076 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.476401 14284 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:03.582492  8890 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:02:03.587486  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.588835  8817 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:03.451691 14307 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.589865  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.591250  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.657480  8894 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:03.592284  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.593530  8891 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:03.593678  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.615607  8820 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:03.659938  8893 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:03.626677  8896 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:03.630425  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 916) took 1417 ms. Trace:
I20260709 13:02:03.661446  8821 rpcz_store.cc:276] 0709 13:02:02.212656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.213554 (+   898us) service_pool.cc:224] Handling call
0709 13:02:02.227191 (+ 13637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.279122 (+ 51931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.630153 (+1351031us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54000203d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:03.630361 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30270848}
I20260709 13:02:03.637107  8815 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:03.645519 14173 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.646474 14284 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.647306 14076 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.649718  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.653682 14040 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.556429 14036 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:03.665367  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.665469  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.667444  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.675684  8824 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.668256  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2342) took 1763 ms. Trace:
I20260709 13:02:03.670047  8821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:03.676906  8821 raft_consensus.cc:606] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:03.677326  8821 raft_consensus.cc:614] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:03.677744  8890 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:03.672109  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.672547  8747 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:03.679247  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.676721  8742 rpcz_store.cc:276] 0709 13:02:01.904435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.314682 (+410247us) service_pool.cc:224] Handling call
0709 13:02:02.314927 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.315112 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:02.315805 (+   693us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.969468 (+653663us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b18001ac4a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:02:03.441162 (+471694us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.667901 (+226739us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.667954 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:03.668016 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.668094 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.668203 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":339968,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4144,"raft.queue_time_us":32953,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":3383,"spinlock_wait_cycles":6116096,"thread_start_us":686141,"threads_started":3}
I20260709 13:02:03.680243  8890 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:03.514753 14063 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.683689  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:03.679222  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 236457us
I20260709 13:02:03.669057 14293 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:03.690111  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.690837  8822 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:03.692636  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.692720  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.693743  8742 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.681370 14036 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.704471  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.706153 14036 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.709398 14145 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:03.709898 14145 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.710250 14145 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.714701  8897 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:02:03.716027  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.716069  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.686965 14311 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.715524  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.719916  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.720782  8895 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:03.725495  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.727705 14312 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.732924  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.728885  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:03.741205  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2387) took 2524 ms (client timeout 1633 ms). Trace:
W20260709 13:02:03.741719  8898 rpcz_store.cc:269] 0709 13:02:01.243281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.243562 (+   281us) service_pool.cc:224] Handling call
0709 13:02:03.737226 (+2493664us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400248954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:03.741114 (+  3888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310464768}
I20260709 13:02:03.747982  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.748805  8898 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:03.755316  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.755966  8898 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.753119 13810 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:03.767704  8740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2333) took 2041 ms (client timeout 1680 ms). Trace:
W20260709 13:02:03.768149  8740 rpcz_store.cc:269] 0709 13:02:01.725943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.816157 (+ 90214us) service_pool.cc:224] Handling call
0709 13:02:03.767458 (+1951301us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54000b5c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:03.767616 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009694464}
I20260709 13:02:03.769531 14307 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.772022 14076 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.773077 14063 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.773569  8898 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:03.774706 14173 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.777997  8822 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:03.779045  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:03.780278  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.769632  8749 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:03.780995 13733 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.781659  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.782397  8821 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:03.783253  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.784011  8821 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:03.784723  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.785634  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.786701  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:03.789417  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2120) took 1002 ms. Trace:
W20260709 13:02:03.790058  8864 connection.cc:256] RPC call timeout handler was delayed by -1.0967832090000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 13:02:03.785231  8821 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:03.780957  8822 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
W20260709 13:02:03.803766  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81228.8us
I20260709 13:02:03.802868  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:03.812660  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.813423 14063 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:03.813674 14076 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:03.813962 14063 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:03.814956  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.789806  8816 rpcz_store.cc:276] 0709 13:02:02.787300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.812078 (+ 24778us) service_pool.cc:224] Handling call
0709 13:02:02.812893 (+   815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.813895 (+  1002us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:02.813939 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.432330 (+618391us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b2c0005b728. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:02:03.432705 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.724744 (+292039us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:03.724793 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.724939 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.789101 (+ 64162us) raft_consensus.cc:1685] finished
0709 13:02:03.789119 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.789349 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":545,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":22632,"spinlock_wait_cycles":17925888,"thread_start_us":291739,"threads_started":1,"wal-append.queue_time_us":324793}
I20260709 13:02:03.801759 13827 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.817701 14229 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:03.818822  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:03.818758 13827 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.819361 14229 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:03.819796 14229 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.821200 13827 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.821549 14115 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:03.821493 14186 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:03.822135 14115 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:03.822721 14186 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:03.823124 14186 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:03.822999 14115 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.824180 13368 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:03.825206 13368 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:03.823995 14186 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.824997 14263 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:03.826318 14186 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.814529 14063 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:03.830847  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 835486us
I20260709 13:02:03.830639 14145 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.831636  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.832696  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.801011 14234 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:02:03.815076 14173 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:03.833829 14304 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:03.834220 14282 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:03.834635 14282 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.834993 14282 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:03.835945 14145 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:03.834240 14304 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.837246 14304 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.837520 14305 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:03.838058 14305 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.840251 14305 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.814476 14076 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.841899  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.842325 14076 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.845474 14123 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:03.846037 14123 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:03.815052  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.825599 13368 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:03.853613  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.854178  9445 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.849191  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.848477  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 942) took 1022 ms. Trace:
I20260709 13:02:03.820616 14229 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.811803 14311 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.812556 14327 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.862795  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.850620 14293 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:03.851459 14326 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.852272 14038 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:03.855540  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:03.870409  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.855962  8747 rpcz_store.cc:276] 0709 13:02:02.826481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.826607 (+   126us) service_pool.cc:224] Handling call
0709 13:02:02.826996 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.678746 (+851750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.848211 (+169465us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400358254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:03.848419 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944092032}
I20260709 13:02:03.871016  8742 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:03.849673  9415 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:03.856314  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:03.871745  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.872396  8740 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:03.872428  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:03.873180  8742 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:03.848470  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:03.874733  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:03.874627  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:03.875485  8747 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.855418  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.857617  8745 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:03.858161  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:03.858618  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.863387  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:03.864444  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.870119  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:03.864522  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.864832  9443 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:03.866958  9444 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:03.867080  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.868130  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.883728  9441 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:03.860404 13813 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:03.861310 14145 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:03.881016  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0524674690000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:02:03.891376  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:03.868871 14048 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:03.909687  9445 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:03.887883  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.901571  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.901731  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:03.911651  8740 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:03.905448 14235 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:03.910813  9446 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.915161  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:03.869223 14327 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.878985 13827 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:03.881266 14229 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.894450 14207 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:03.920817 14327 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.921131 14289 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.921629 14289 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.921890  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.922488  8740 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:03.923627 14289 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.923759  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.860910 13839 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:02:03.885941 14173 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:03.886933 13733 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:03.887513 14326 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:03.885146 14277 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:03.873456 14293 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.927942  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.939491  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:03.924222 14063 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:03.900079 14151 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:03.865929 14311 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:03.954268  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62975.9us
I20260709 13:02:03.917248 14235 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:03.958051  8747 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:03.981606  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.964658  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.982391  8896 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 10.
I20260709 13:02:03.965045  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.966377  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.967176  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.967484  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.968384  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2128) took 1179 ms. Trace:
I20260709 13:02:03.985250  8815 rpcz_store.cc:276] 0709 13:02:02.789024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.804574 (+ 15550us) service_pool.cc:224] Handling call
0709 13:02:03.968189 (+1163615us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:03.968325 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904076032}
I20260709 13:02:03.929792 14048 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:03.905359 14167 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:02:03.912750 14145 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.970476  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:03.988103  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.988780  8894 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:03.983959  8890 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:03.886221 14194 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:02:03.977572  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:03.989588  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:03.969635 13827 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:03.948868 14182 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:03.988363  8898 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.924961 14234 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:03.955739 14207 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.991303  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:03.929523 14053 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:02:03.948449 13733 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:03.988788  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:03.953589 13810 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:03.918789 14111 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:02:03.977247 13578 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.001000  8747 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:03.992398  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:03.970683 14326 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:03.981401 14173 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:03.913611 14313 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:03.987514 14151 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:03.995782 13827 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.967290 14305 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:02:04.014814  8898 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:03.996491 14123 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:03.947758 13368 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.002018 14234 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.061480 14234 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.062081 14311 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:04.034682 14327 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:04.015744 13578 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:04.063354 14327 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.063700 13578 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.064173 14327 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.017760  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2089) took 1078 ms. Trace:
I20260709 13:02:03.997705 14293 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.065442  8741 rpcz_store.cc:276] 0709 13:02:02.939642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.021991 (+ 82349us) service_pool.cc:224] Handling call
0709 13:02:03.307639 (+285648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.319729 (+ 12090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.017553 (+697824us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400359150. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:04.017705 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651923712}
I20260709 13:02:04.065436 14327 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.021610 14326 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.068428 14151 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.026975 14296 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:04.006880 13733 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.035066 14304 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.034934 14053 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:02:04.056571 13827 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:04.069276  8749 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:04.079809 14282 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:04.060837 14342 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:02:04.060954 13368 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.023173 14173 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:04.031876 13912 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.059279 14300 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.072304  9442 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:04.083293  9447 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
W20260709 13:02:04.098862  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.006186 14207 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:04.110126  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 121415us
I20260709 13:02:04.107915 14032 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:04.121011  9448 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:04.072085 14304 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.082327 14327 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:04.132144  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:04.074682 14234 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.139627 14063 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.141808  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.142807 14234 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.143316 14168 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.143469  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.144345  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.145516 14173 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.147552 13709 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.148168 14207 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:04.147944 13578 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.123809 14300 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.148900 14300 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.149446 14286 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.150421  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.151098  8823 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:04.152061  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.152643  8817 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:02:04.108393 14145 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.155509  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.156100  8818 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:04.134635 14327 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.128167 13912 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.156796  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:04.157701  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.158394  8818 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:04.158538 14327 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.106403 14048 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.160073  8823 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.160874 14032 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:04.161357 14032 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.070292 14296 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.162397  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:04.162699  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.163142  8822 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:04.134919  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 198916us
I20260709 13:02:04.164183  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:04.164202  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.166005  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:04.167126  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.168247  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.170135  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.163635  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.178949  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:04.136986 14313 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:04.182740 14293 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:04.182806 13709 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.143421 14234 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.204241 14313 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:04.186789 14036 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:04.230326 14235 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.202072 14349 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.231118 13733 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.282356  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:04.365866 14173 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:04.366990 14173 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.368832 13733 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.184252  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:04.293969  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 309263us
W20260709 13:02:04.319058  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.384333  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 199687us
I20260709 13:02:04.246296 14235 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.245944 14282 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.385653  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.234191 14293 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:04.387260 14036 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:04.245148 14296 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.243317 14300 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.387959 14036 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:04.390348 14036 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.392607 14300 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.234248 14313 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.240638 14349 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.395737 14349 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.350914  8890 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:04.351528 13578 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.353873  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1088) took 1290 ms. Trace:
I20260709 13:02:04.397821  8894 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:04.397073  9440 rpcz_store.cc:276] 0709 13:02:03.063486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.063683 (+   197us) service_pool.cc:224] Handling call
0709 13:02:03.064139 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.835837 (+771698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.835871 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.836325 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.085778 (+249453us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:04.085828 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.085934 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.353130 (+267196us) raft_consensus.cc:1685] finished
0709 13:02:04.353182 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.353574 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":771231,"raft.queue_time_us":776466,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":51,"thread_start_us":249118,"threads_started":1,"wal-append.queue_time_us":371025}
I20260709 13:02:04.243330 13827 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.388451 14293 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.401787 13827 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.355767 14145 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:04.402684 14145 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:04.404038 14145 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:04.291219  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 127478us
I20260709 13:02:04.249011 14032 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.405684 14048 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:04.406670 14048 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.407043  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.407734 14032 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.408288  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.371825  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.357548 14353 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:04.405018  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.411041  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.411659 13709 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:04.414799  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.411159 14353 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.409258 14048 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:04.412083 14151 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:04.449568  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.449438 14234 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:04.450585 14234 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.452257  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:04.454550  8823 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:04.455643 14353 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.434007 13709 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:04.458983 13709 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.456823 14355 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:04.460099 14355 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:04.460539 14355 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:04.461516  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 74471.1us
I20260709 13:02:04.462661  8787 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:04.463758 14353 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:04.464994 14353 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:04.443866 14151 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.465860 14151 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.466579 14353 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:04.475225  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61727us
I20260709 13:02:04.475591 14296 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:04.494161 14296 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.498728 14296 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:04.499017  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 321833us
I20260709 13:02:04.500528  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:04.503077  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 117779us
W20260709 13:02:04.501488  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.509053  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.498193 14235 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:04.517813 14235 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.522801 13813 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:04.523705  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.558284 14350 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.559098 14350 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.562670 14214 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.563611 14214 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.565943 14167 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.566664 14167 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:04.569046  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.570552  8815 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:04.570668  8822 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:04.571496  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.572547  8820 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:04.574023  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.575027  8816 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:04.575994  8893 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:04.576725 14167 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.578372 14214 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.578850 14296 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.583456  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:04.585810 14053 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:04.586432  8821 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:04.586853 14350 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.587015 13813 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.587455 13813 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:04.587836 13813 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.589499  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.592096  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:04.592475  8820 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:02:04.593146 14214 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.593976  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:04.594820 14350 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.595604  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:04.596504 14350 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.589671 14053 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:04.538308 14234 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:04.597918  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 59323.8us
I20260709 13:02:04.597209 14053 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.599798  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:04.601455  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.581157 14296 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.610370  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.630565 14151 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.633617 14151 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.638306  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:04.639014 14214 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:04.645606 14033 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:04.646998 13368 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:04.648240 14284 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.648748  8817 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:04.630475 13813 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.653961  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:04.663655  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 153580us
I20260709 13:02:04.666684 13827 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:04.667174 14235 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.667812 14235 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:04.667558 13827 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:02:04.671437  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.671494 14115 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.676033  8863 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:04.676414  8820 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:04.656483 14214 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:04.674770 14033 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:04.686028  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:04.676959 14300 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:02:04.682493 13368 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.687337 13827 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.687652 14214 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:04.663767 14036 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:04.699537 13912 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.012s
I20260709 13:02:04.700451 14312 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:04.701434 14313 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.701851 14186 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:04.703287 14186 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:04.703732 14351 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:04.703960 14313 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:04.704336 14186 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:04.704900 14186 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.705185 13839 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:04.704306 14351 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.657843 14053 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.706609  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:04.708017  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:04.708931 14378 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:02:04.691026 13813 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.717957  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:04.722445 14067 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:04.723030 14151 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:04.723209 14067 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:04.723693 14151 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:04.724810 14151 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.726339  8741 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:04.726217 14351 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.727818 14351 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:04.728114  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.729210  8747 raft_consensus.cc:2410] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:04.714619 13839 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.731249  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.732677  8740 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:04.733206 14307 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.733671 14307 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.734100 14307 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.734625 14053 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:04.734815  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 858355us
I20260709 13:02:04.724534 14067 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.737007  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.717099 14235 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:04.749990  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 152792us
I20260709 13:02:04.748908  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.755223  8816 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.751556 14048 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.763324 13839 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.764488 14294 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:04.764665 13839 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.764694  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.770035  8740 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:04.770848  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.751996 14355 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.781538 14040 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:04.782171 14040 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:04.782618 14040 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.785379 14048 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.797030  8821 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:04.765003 14033 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.801580  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.775234 14313 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:04.813380  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:04.821807  8893 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:04.820487  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:04.823549  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 86702.2us
W20260709 13:02:04.824533  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67555.4us
I20260709 13:02:04.816381 14036 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.824240  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.830551  8741 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:04.802143 13368 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.828814  8740 pending_rounds.cc:85] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 8
I20260709 13:02:04.832379  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:04.832737  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72908us
W20260709 13:02:04.837563 14379 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:02:04.833757  8740 pending_rounds.cc:107] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:02:04.838435  8740 raft_consensus.cc:2889] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:04.841225 14173 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.842710  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:04.843438  8823 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:04.843778 14357 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:02:04.820101 14206 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.858129  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:04.858878  8815 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:04.878597 14294 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.877633  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:04.875519  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:04.814363 14186 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:04.883493  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.886639  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.887419  8893 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:04.905509  8890 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:04.906082  8890 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:04.907275  8890 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.908537  8890 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:04.918322  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.919299  8747 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:04.924774 14048 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:04.925555 14048 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.926254 14048 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:04.930254  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:04.971822  8788 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:04.928970 14048 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.974974  8815 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:04.980610  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:04.889394 14173 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:04.985029  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 97903.3us
I20260709 13:02:04.889726 13813 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
W20260709 13:02:04.990360  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 91422.7us
I20260709 13:02:04.991189  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:04.994241 14067 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.994414  8747 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:04.997212 14386 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:02:04.985621 14355 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:04.981503 14206 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:04.977257 14357 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:05.013530  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.000777 14033 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:05.017272  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.004138 13827 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:05.021389 14268 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:05.025859 13827 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.026329 14268 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:05.026755 14268 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:05.026935 13827 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.028163 13827 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.002835 14173 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.029799 14307 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.030622 14173 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:04.972899 14294 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.032254 14307 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:05.032948 14294 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:05.011233 14206 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.033842 14206 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.037714  8822 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:05.020219  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.041550  8747 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:05.027688 14268 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:05.043642  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61666.6us
I20260709 13:02:05.045176  8891 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:05.052063 13709 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.592s	user 0.014s	sys 0.000s
I20260709 13:02:05.052665 13709 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:02:05.052132 14387 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:02:05.054414 14268 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.056347  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.058346 14307 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:05.058919  8748 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:02:05.060835 14186 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:05.075469 14307 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.063330  8746 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:05.063740  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:05.070501  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 60418.5us
I20260709 13:02:05.062144  8745 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:05.078507 14294 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:05.078531  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:05.080693 14355 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:05.081254 14355 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:05.082257 14355 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.078032 14307 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.094789 14040 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:05.101521  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 84255.5us
W20260709 13:02:05.114948  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.129552 13709 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.131953  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.133304  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.133850  8819 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.136734  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:05.137848  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.145238  8749 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:05.145794  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.163508  8890 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:05.166394  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.167025  8824 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.158252  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.157356  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:05.168998  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82781.3us
I20260709 13:02:05.179725  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.185500  8816 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:05.186273  8815 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:05.188629  8822 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:05.189728 14076 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:05.190306 14384 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:02:05.190410 14076 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.192795 14076 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.192749  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:05.211927 14214 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.005s
I20260709 13:02:05.212586 14214 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.213786  8820 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:05.214926 14214 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.215751  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.216305  8894 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:05.216956  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.217638  8820 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:05.218334  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.218933  8820 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:05.219565  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.220531  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.221457  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.222379  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.223657  8820 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:05.232016  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.232712  8819 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:05.233984  8823 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:05.236963  8816 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: [14.12-15.13]
I20260709 13:02:05.237560  8747 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:05.239303  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.239744 14040 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:05.240118  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.240832  8818 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:02:05.242698  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 73007.3us
I20260709 13:02:05.245419  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.252144  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.254582  8819 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:05.244163 14115 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.007s
I20260709 13:02:05.266006  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.265829 14053 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:05.275406  8897 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:05.223923 14206 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.289249  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:05.295704  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 191612us
W20260709 13:02:05.297698  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58499.6us
I20260709 13:02:05.272289 14040 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.276407 13813 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:05.309583 14313 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:05.309690 14036 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:05.313397 14036 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:05.313956 14036 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:05.309574  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 116999us
W20260709 13:02:05.319116  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53302.8us
I20260709 13:02:05.332068  8864 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.303591 13813 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.317996 14389 log.cc:927] Time spent T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.342s	user 0.003s	sys 0.000s
W20260709 13:02:05.346269  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 165188us
I20260709 13:02:05.348971  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.353585  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.304143 14053 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:05.338090 14115 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.338914 13813 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.385640  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:05.403970  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 64265.7us
W20260709 13:02:05.413937 14395 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 13:02:05.418080  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2427) took 1451 ms (client timeout 1843 ms). Trace:
W20260709 13:02:05.418568  8897 rpcz_store.cc:269] 0709 13:02:03.966263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.966893 (+   630us) service_pool.cc:224] Handling call
0709 13:02:05.417830 (+1450937us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540015de54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:05.417985 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2968363776}
I20260709 13:02:05.327133 14040 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:05.423923  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76015.4us
I20260709 13:02:05.337774 14351 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:02:05.403268 14115 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.447772  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 141230us
W20260709 13:02:05.452638 13839 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.003s
I20260709 13:02:05.483949  8819 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:05.471907 14115 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.493371 14357 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:05.490651 14294 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.497989 14206 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.475455 14351 consensus_queue.cc:1243] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:05.509099  8824 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:05.510072  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.512589  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.513219  8821 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:05.515646  8821 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:05.501744 14357 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.516139  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.521967  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.504086 14048 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.524958  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.525648  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:05.535745  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.527813  8741 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:05.538837  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.539860 14393 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:05.524230 14357 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:05.540701 14393 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.541778  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.541678 14294 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.543989 14294 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:05.535334  8748 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:05.545362  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.546125  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.546499  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.547190  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.547863  8748 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:05.539069 14040 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.541051  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:05.550614 13368 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:05.550809 14398 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:05.550580  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.549002  8748 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:05.547708  8742 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:05.550482 14404 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:05.503980 14307 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.551295 14398 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.552127  9440 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:05.551139 13368 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.557412  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:05.555230  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 102867us
I20260709 13:02:05.528083 14351 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.565943  8741 raft_consensus.cc:2368] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
W20260709 13:02:05.561743  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 150406us
I20260709 13:02:05.563659  8746 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:05.564604  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.570894  8822 raft_consensus.cc:2368] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15.
I20260709 13:02:05.569269  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.571949  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.557955 14398 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:05.554376 14404 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.574553  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:05.578725  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 245824us
I20260709 13:02:05.580360  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:05.534307 14053 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:05.581707  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.581913  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.582698  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.583369  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.583783  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.584022  9449 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:05.584362  9445 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:05.585134  9444 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:05.585815  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.581990  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.586810  8742 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.587066  9443 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.588717  9440 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:05.594086  9447 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:05.593861 13368 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.593904  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.595396 13368 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:05.595825 14394 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.357s	user 0.003s	sys 0.000s
I20260709 13:02:05.597553 14391 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:05.598384  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.599963  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.601087  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.601684  8743 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:05.602864  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.599469  9447 raft_consensus.cc:2410] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:02:05.612679  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43657.6us
I20260709 13:02:05.544060 13813 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.616894  9440 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:05.623203  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.503834  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:05.635150  8743 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:05.639393  9443 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:02:05.632830  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58403.1us
I20260709 13:02:05.642935  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:05.643767  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.642963  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.649828  8746 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:05.598428 14313 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:05.653589  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.654300  9447 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:05.655371  9445 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:05.657612 14412 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:05.660774 14393 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.572654 14151 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:02:05.687508 14151 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:05.687894 14151 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:05.688874 14151 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.690172 14151 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:05.691084 14410 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:05.691771 14410 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.639642 13912 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:05.693948 13912 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:05.694051 14410 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:05.662719  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 67635.6us
I20260709 13:02:05.694990 13912 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.696535 14402 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 2 to 10, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:05.698161  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.699098  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.699782 14067 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:05.666500 14404 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.667724 13827 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.670188  8748 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:05.577827 14076 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:02:05.683732  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 319479us
I20260709 13:02:05.685498  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.570815 14048 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:05.714030  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 133797us
I20260709 13:02:05.671324 14313 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.710021 14067 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:05.643869 14040 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.722103  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.706815 14268 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.745788  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:05.723541 14033 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.721145 14067 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:05.714061 14053 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:05.728977 14268 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.758070  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.720772 14391 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:02:05.758658 14404 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:05.762894 14067 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:05.746596 14412 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:02:05.773543  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:05.766804 14268 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:05.772130  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 119556us
I20260709 13:02:05.783653  9442 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:05.785486 14393 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:05.785957 14393 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:05.786897 14393 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.757833 14040 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.788277 14393 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.789616 14040 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.782450  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:05.794346  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.795277  8890 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:05.792379  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:05.799988  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.801061  8892 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:05.804873  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.805543  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.805781  8896 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:05.770299 14033 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.806352  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.808460 14033 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.809036 14391 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.806380  8894 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:05.805903  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.817370  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.823302  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.824185  9447 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:05.780215 14307 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:05.829573  8824 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:05.825129  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.831527  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.832437  8819 raft_consensus.cc:2410] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:05.833871  9440 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:05.835337  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.836959  9441 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:05.826310 13813 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:05.839186 13813 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.839646 13813 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.840816  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.840888  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:05.846151  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:05.776147 14053 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.841588  9448 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:05.811239 14036 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:05.850417  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.850591  9447 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.833022  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:05.855545  9449 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.867372  8742 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:05.874210  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:05.865115  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72819.3us
I20260709 13:02:05.881489 14405 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:05.881731 14053 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:05.881985 14405 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:05.882479 14405 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.885445 14424 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:05.885639  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.886097 14424 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.886541  8892 raft_consensus.cc:2410] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:05.887036 14424 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.889789 14173 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:05.891954 14409 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:05.892097  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:05.892547 14409 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:05.893638 14409 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.895454  9441 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:05.900280 14398 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.903998  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:05.904860 14425 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:05.871403 14415 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:02:05.905639 14413 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:05.906729  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:05.912681  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 141845us
I20260709 13:02:05.859321 14036 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.860517 14399 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.911808  9447 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:05.912334  9446 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:05.796603 14313 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.915663  9446 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:05.871179 14421 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.926226  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.926963  8894 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:02:05.926923  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:05.884755 14268 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:05.929483  9449 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:05.939975  9444 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:05.941659  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 195973us
I20260709 13:02:05.936530 14420 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:05.904829 14053 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:05.949914  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70212.9us
W20260709 13:02:05.956516  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 164871us
I20260709 13:02:05.916704 14412 raft_consensus.cc:2764] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:05.961516  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.871011 14407 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.170s
I20260709 13:02:05.965947  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.929400 14351 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.941059 14391 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.943463 14421 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:05.959877  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.971829  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:05.973644  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:05.974287  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:05.977524  8889 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:05.984102  8824 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:05.992527  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:05.992964  8822 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:02:05.995605  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 68752.5us
W20260709 13:02:05.998184  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.998940  8817 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:02:06.004695  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44924.2us
I20260709 13:02:05.966269 13709 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.007118  8741 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:02:06.011394  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 60242us
I20260709 13:02:05.975173 13912 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.967017 14186 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:06.014587  8818 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:05.926344 14399 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.017702  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:06.013182  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:05.983033  9446 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.029145 14411 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.029404 14398 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:05.985803 14421 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.993140 14355 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.966703 14307 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:06.041906  8894 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:02:06.041271  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:06.049796 14427 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:02:06.017249 14313 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.015167 13709 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:06.036872  8740 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:05.981472 14053 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:06.055017  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50861.9us
I20260709 13:02:06.031636 14206 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:06.069335  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:06.070418  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56449us
I20260709 13:02:06.041596 13813 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.026300 13912 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:06.074183  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.077800 14435 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.078051  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:06.055269  8894 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 7
I20260709 13:02:06.081113  8896 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:06.114284  8894 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:06.114655  8894 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:06.082259 14403 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.513s	user 0.000s	sys 0.003s
I20260709 13:02:06.067765 13709 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.087239 14268 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.087409 14053 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.093443 14048 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:05.988554 14391 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.106709  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.107909  8745 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:06.094058 14206 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:06.003546 14357 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.123898  9446 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:06.104863 14115 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
W20260709 13:02:06.126577  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48617.2us
I20260709 13:02:06.044652 14033 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 13:02:06.235881  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122282us
W20260709 13:02:06.214951 14427 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:02:06.224584  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:06.116485 13912 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.264570  8889 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:06.260991  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.053376 14411 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:06.200392  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.285341  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 313613us
W20260709 13:02:06.286252  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 179663us
W20260709 13:02:06.144487  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.296818  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72333.5us
I20260709 13:02:06.297897  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:06.295410 13839 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:06.320971  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:06.321923  8824 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.123265 14425 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:06.208845 14398 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 13:02:06.331614  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:06.287338 14410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.007s
I20260709 13:02:06.214357 14206 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:06.219386 14115 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.225394 14048 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:06.348109  8892 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:06.349588  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:06.352037  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62737.8us
I20260709 13:02:06.352674  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:06.339357 14410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.009s	sys 0.007s
I20260709 13:02:06.328896 13839 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:06.368211  8789 leader_election.cc:400] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:02:06.368589  8789 leader_election.cc:403] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:02:06.366290  8819 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:06.213831 14435 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.371392  8824 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:06.339066 14441 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.380223 14401 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:02:06.380945 14433 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.266s
I20260709 13:02:06.382246 14439 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:02:06.338589 14355 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:02:06.370915 13839 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:06.383383 14418 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 13:02:06.384806  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:06.385972 14437 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:02:06.387255 14422 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.389991 14399 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.391413 14357 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:06.389037 14067 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.000s
W20260709 13:02:06.444242  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.353493 14407 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:06.353821 14048 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:06.474319  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 477717us
W20260709 13:02:06.383014 14426 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:02:06.345337 14350 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:06.481447 14433 consensus_queue.cc:1243] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:06.485682 14439 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.487066 14294 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.487694 14115 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:06.488157 14436 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:02:06.490111  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.349190 14053 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.492362 14151 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.003s
I20260709 13:02:06.465574 14441 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:06.540851  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 156146us
I20260709 13:02:06.545879 13813 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:06.491962 14410 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.478634 14036 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.541481  8788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.018s
I20260709 13:02:06.546540  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:06.552382  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 217993us
W20260709 13:02:06.571314  8788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.002s	sys 0.018s
W20260709 13:02:06.571653  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 561009us
I20260709 13:02:06.572201  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:06.572537  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.544417 14048 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.543601 14268 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:06.561239 14443 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:06.561393 14445 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13.
I20260709 13:02:06.562410 14294 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.562989 14053 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.563364 14454 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:06.594792 14294 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.595201 14454 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.596102 14294 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.596666 14294 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.583004 14268 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:06.597754  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 51292.7us
I20260709 13:02:06.598500 14268 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.599022  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.599717 14268 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.569000  8891 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:06.556140  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.542037 14067 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.584321  9446 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:02:06.580142 14151 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.603705  9446 pending_rounds.cc:85] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 12
I20260709 13:02:06.624651  9446 pending_rounds.cc:107] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:02:06.625054  9446 raft_consensus.cc:2889] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:06.624785  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.568828 14441 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.492246 14425 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.561892 14444 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
W20260709 13:02:06.662976  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.664902  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:06.666599  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.627712 14350 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:06.687615 14350 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.687775 14454 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.688153 14350 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:06.688565 14454 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:06.688990 14454 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:06.714888 14454 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:06.716271 14393 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.928s	user 0.010s	sys 0.001s
I20260709 13:02:06.672766 14448 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:06.663650 13709 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 13:02:06.717705 13709 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.673275 14460 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:06.681437 14450 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:02:06.662813 13827 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.951s	user 0.014s	sys 0.002s
W20260709 13:02:06.726368  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 126265us
I20260709 13:02:06.727602  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:06.685742 14067 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.640079 13813 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:06.686858 14053 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:06.693815  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.694490  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.601070 14424 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 13:02:06.695000 14429 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:06.695329 14449 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
W20260709 13:02:06.695824 14405 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.813s	user 0.007s	sys 0.001s
I20260709 13:02:06.696206 14173 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:06.700582 14438 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
I20260709 13:02:06.702739 14435 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.586071 14115 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:02:06.579995 14036 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.594859 14432 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:02:06.620504 14443 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.666100 14411 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:06.717646 14448 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.673735 14455 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:06.580576 14355 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:02:06.570055 14439 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:06.693136 14313 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.001s
I20260709 13:02:06.772832  8893 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:06.769761 14460 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:06.778473  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 480717us
I20260709 13:02:06.792203 14449 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:02:06.769917 13827 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.771557 14186 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.851171 13368 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:06.854182 13368 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:06.853960 14186 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.854952 14036 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.855085 13368 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.856213 14186 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:06.857811 14406 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:02:06.860265  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.861575  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.861852 14447 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:06.862259 14447 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:06.862711  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.863255  8893 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.863124 14447 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.864693 14447 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.866042 14294 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.672243 14151 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:06.783056 13912 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:02:06.804989  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.809790 14425 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:06.803655  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 228069us
I20260709 13:02:06.778132 13813 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:06.810350  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 96058.4us
W20260709 13:02:06.782202  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 207503us
W20260709 13:02:06.849895  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 285054us
I20260709 13:02:06.850294  8890 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 13:02:06.810214 14404 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.034s	user 0.012s	sys 0.000s
I20260709 13:02:06.867910  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 950) took 1158 ms. Trace:
I20260709 13:02:06.768746 14206 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.903566  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2162) took 1071 ms. Trace:
I20260709 13:02:06.788086 14405 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.872148  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:06.889555  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:06.892086  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 286786us
I20260709 13:02:06.921232  8893 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:06.903981  8815 rpcz_store.cc:276] 0709 13:02:05.814800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.814912 (+   112us) service_pool.cc:224] Handling call
0709 13:02:05.847532 (+ 32620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.857443 (+  9911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.857490 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.867768 (+ 10278us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.953230 (+ 85462us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:05.953273 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.953358 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.699807 (+746449us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:06.867110 (+167303us) raft_consensus.cc:1685] finished
0709 13:02:06.867136 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.886316 (+ 19180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":15740,"spinlock_wait_cycles":6807424,"thread_start_us":36534,"threads_started":1,"wal-append.queue_time_us":347073}
I20260709 13:02:06.872561  8897 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:06.856173 13368 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.905833  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:06.909678  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:06.910010  8749 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:06.910372  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:06.911000  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:06.911645  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:06.912197 14443 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.914656 14448 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.916098 14151 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.917549 14439 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.917889  8898 raft_consensus.cc:2410] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:06.944240 14186 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.944792 14186 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.947233 14404 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:06.944240  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.949631  8893 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:06.951599  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:06.901727  9443 rpcz_store.cc:276] 0709 13:02:05.709866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.710344 (+   478us) service_pool.cc:224] Handling call
0709 13:02:05.710765 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.742785 (+ 32020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.742814 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.753968 (+ 11154us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.917403 (+163435us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:05.917446 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.917510 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.867703 (+950193us) raft_consensus.cc:1685] finished
0709 13:02:06.867726 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.867850 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7587,"raft.queue_time_us":12716,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":13296,"thread_start_us":133540,"threads_started":1,"wal-append.queue_time_us":704662}
I20260709 13:02:06.830065 14455 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:06.805824 14435 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:02:06.866915 14444 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:06.924789  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 52754.1us
I20260709 13:02:06.868491 14048 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.959578  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 953) took 1005 ms. Trace:
I20260709 13:02:06.928273  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.928893 14268 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:06.929366 14313 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.931201  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:06.959988  8889 rpcz_store.cc:276] 0709 13:02:05.954560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.989824 (+ 35264us) service_pool.cc:224] Handling call
0709 13:02:06.017522 (+ 27698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.284997 (+267475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.959344 (+674347us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54001806d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:06.959519 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562334080}
I20260709 13:02:06.964015  8746 raft_consensus.cc:2410] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:06.968199  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:06.972553 14461 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:06.972772  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:06.973415  9447 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.973590 14464 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.973904 14466 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:02:06.974238 14464 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:06.976413 14464 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:06.973250 14461 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.931216 14115 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.991920  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.993655  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:06.994886 14435 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:06.995309 14435 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:06.996172 14435 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.997674 14435 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.982725  9447 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:06.966578  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:06.872219 13912 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.001890  9444 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:06.977886  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.003204  8748 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:06.964301 14421 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.892s	user 0.007s	sys 0.004s
I20260709 13:02:07.004527  9448 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:06.957556 14441 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.904496 14425 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:07.010859  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.012247 14463 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.012768 14463 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.013615 14452 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.014146 14452 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.014603  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 950) took 1061 ms. Trace:
I20260709 13:02:07.015195 14463 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:07.016019 14408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.022s	sys 0.034s
W20260709 13:02:07.016407 14408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.022s	sys 0.034s
I20260709 13:02:06.875855 14350 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.003248 14464 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.006886 14067 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:07.007077 14445 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:06.961433 13827 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.964005 14411 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.014976  8898 rpcz_store.cc:276] 0709 13:02:05.953454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.953576 (+   122us) service_pool.cc:224] Handling call
0709 13:02:07.014382 (+1060806us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b5400094554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:07.014537 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866170112}
I20260709 13:02:06.954083 14455 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.964892 13709 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:06.963745 14432 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.977638 14036 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:07.033213  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.036408  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:06.948416 14404 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:06.972927 14444 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.072981  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.048062  9440 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:07.034359  8862 connection.cc:256] RPC call timeout handler was delayed by -1.2486983180000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:02:06.979247 14443 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:02:07.062372  8746 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:07.063426  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:06.978117 14313 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.038580 14464 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.977896 14268 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:07.061455 14460 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:06.977921 14353 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:02:06.978960 14439 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
W20260709 13:02:07.084817 14430 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.865s	user 0.001s	sys 0.017s
I20260709 13:02:07.041702 14452 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.085343  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.146637  8743 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:07.009176 14402 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:02:07.086054  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49791.6us
W20260709 13:02:07.055306 13839 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.527s	user 0.003s	sys 0.007s
I20260709 13:02:07.009603 14115 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.050307 14391 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.054760 14411 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.075287 14475 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.080404 14040 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.161492 14040 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.129982 14036 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:07.151113 14444 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.091428 14406 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:07.141139  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2382) took 1149 ms. Trace:
I20260709 13:02:07.139516 14268 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.163447  8748 rpcz_store.cc:276] 0709 13:02:05.991553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.991661 (+   108us) service_pool.cc:224] Handling call
0709 13:02:06.012823 (+ 21162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.021644 (+1008821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.140815 (+119171us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:07.141037 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277461376}
I20260709 13:02:07.150245  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.163580  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.031452 14421 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.155273  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.157730 14033 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:07.146854 13912 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.159364 14115 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.160964 14411 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.133692 14424 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:02:07.163451 14406 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.167819  8820 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:07.197827  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.198669  8745 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.087777 14206 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.214761  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.215876  8820 raft_consensus.cc:2410] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:02:07.156140 13839 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.219112  8889 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:07.219808 13839 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.220888  8894 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:07.066972 14445 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:07.173341  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.158710 14441 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.185801  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:07.236560  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 258751us
I20260709 13:02:07.135540 14464 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.054416 14067 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.082705 14454 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.161883  8815 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:02:07.190339  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.193796  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.194007  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:07.247852  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.043084 13813 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.198757  8819 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:07.136605 14353 consensus_queue.cc:1243] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:07.199834  8816 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:07.197837 14033 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.201725 14478 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:07.202021 14406 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.092025 13827 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.202562  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.185137 14402 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:07.203249 14442 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.268092 14402 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.268524 14402 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.268268 14442 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.270429 14442 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.271324  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.271732  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.271922  8820 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.203624 14439 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:07.272608 14439 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.274721 14439 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.275302  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.151242 14460 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.163010 14036 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.205606 14313 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.206121 14040 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.210920 14421 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.212162 14350 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.226503  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.170768 14447 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:07.244657  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:07.332185  8740 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.247836  8746 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:07.247215  8817 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:07.195291 14425 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.250195 14391 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:02:07.082047 14432 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.171079 14475 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.262182  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:07.344748  8742 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:07.258637 14033 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.263847  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.264680 14461 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.243213 13912 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:07.267586 13827 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:07.276502  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2640) took 1501 ms (client timeout 1992 ms). Trace:
W20260709 13:02:07.355856  8897 rpcz_store.cc:269] 0709 13:02:05.779489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.790439 (+ 10950us) service_pool.cc:224] Handling call
0709 13:02:07.276326 (+1485887us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400248454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:07.276440 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450695552}
W20260709 13:02:07.276885  8896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2647) took 1465 ms (client timeout 1731 ms). Trace:
I20260709 13:02:07.204406 14404 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.280946  8820 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:07.283473  8893 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:07.274744  8898 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:07.257356 14405 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.260839 14478 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.288518  8821 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.290419  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.366122  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.291879  8824 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:07.277508 14048 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.370012  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:07.340960  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.285450 14307 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:07.341144  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.342957  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.288386 14476 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.346308  8740 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:07.377524  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.377957  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.379079  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.380174  8740 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:02:07.288679  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 207186us
W20260709 13:02:07.348945  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.382829  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.383307  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.353457  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.374377  8746 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:07.373615  8817 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:07.332376 14447 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:07.356855  8896 rpcz_store.cc:269] 0709 13:02:05.812045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.817046 (+  5001us) service_pool.cc:224] Handling call
0709 13:02:07.276740 (+1459694us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003bd154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:07.276824 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427984640}
I20260709 13:02:07.369498  8818 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:07.369589  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.369581  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:07.393270  8897 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.393361  8742 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.361868 14206 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.395793  8740 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:07.395891  8824 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:07.397112  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.397650  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.393885  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.398240  8896 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:07.398268  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.399048  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 958) took 1156 ms. Trace:
I20260709 13:02:07.351351  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.399432  8896 rpcz_store.cc:276] 0709 13:02:06.242921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.397420 (+1154499us) service_pool.cc:224] Handling call
0709 13:02:07.398997 (+  1577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.400302  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:07.400050  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48817.2us
I20260709 13:02:07.401005  8896 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:07.403306 14421 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:07.403753 14421 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.404111 14421 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:07.373656 13912 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.369580 14478 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:07.409463  8742 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
W20260709 13:02:07.410009 14479 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:07.410298  8746 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [14.9-17.11]
I20260709 13:02:07.370481 14040 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:07.410485  8818 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:07.411705  8818 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:07.358938 14350 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:02:07.472129  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59597.2us
I20260709 13:02:07.337591 14445 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:07.485436  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 74039.4us
I20260709 13:02:07.394081 13839 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.474968  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 76773.9us
W20260709 13:02:07.487414  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.489099  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.490026  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.410907 14040 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.416026 14313 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.495570 14313 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.498337 13912 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.499147 14313 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.500052 14455 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.471345 14115 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:07.501381 14357 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:07.501521 14408 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:07.501840 14115 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.503896 14115 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.489943  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.510551 13839 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.512163 14411 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.512573 14411 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.512812 13839 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.394883 14476 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.514391  8816 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:07.516769  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:07.402170 14206 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.391222 14307 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:07.518688 14483 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:02:07.520498 14357 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.521052 14357 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.521644  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.523098 14357 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.523412 14351 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:07.523667 14357 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:07.524080 14404 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.524816 14408 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:07.525358 14482 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:02:07.512918 14411 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.526865  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.527295 14067 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.485998 14425 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:02:07.524897 14404 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:07.530598 14307 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.535595 14432 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:07.535939 14455 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.513553  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.543903  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.544549  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.599094  8742 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:07.545661 14454 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.546947 14475 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.543921  8749 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:07.524147 14351 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:07.571110  8897 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:07.574295  8820 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.584188  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.608970  8896 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:07.583498 13839 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:07.623463  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 109996us
I20260709 13:02:07.570436 14313 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:07.628408  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.623942  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.632390  8897 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:02:07.633729  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110455us
I20260709 13:02:07.637377  8896 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:07.636521  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.629570 13813 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:07.648268  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.648921  8897 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:07.646472  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.651518  8896 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.656823  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.661670 14476 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.671375  8896 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:07.674399 14206 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.674055  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.676999  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.679311  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.679404  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.680531  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:07.679471  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.684777  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.685961  8646 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:07.688167 14186 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.690111  8646 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 5 to 8, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 8 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:07.693229 14033 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.696226  8818 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.700730 14399 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:07.701260 14399 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:07.702486 14399 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.704182 14399 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.706030  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2440) took 1028 ms. Trace:
I20260709 13:02:07.706552  8893 rpcz_store.cc:276] 0709 13:02:06.678085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.993422 (+315337us) service_pool.cc:224] Handling call
0709 13:02:07.705788 (+712366us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540032b254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:07.705952 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615607168}
I20260709 13:02:07.709662 13827 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:07.710763 13827 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:07.711108 13827 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:07.714284  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.632483 14313 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.648113 14408 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:07.714792 13827 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.730427 14053 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:07.737043  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:07.737641 14412 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:07.738420  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:07.747931  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.758695  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.763435 14433 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.778566  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98097.8us
I20260709 13:02:07.781015  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:07.825255  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44301.7us
I20260709 13:02:07.652455 13813 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:07.791695  8743 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:07.823781  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.832093  8820 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
W20260709 13:02:07.825620  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.792646  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.688810 14186 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:07.832855  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.822962  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.839900  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.834378  8897 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:07.851408  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.852306  8823 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:02:07.853915  8819 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:07.769029 14408 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:07.856088  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.787670 14475 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.794947 14485 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:07.779810 14053 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:07.860059  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.861322  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.864355  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.795547 14391 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:07.833439 14405 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:07.865569  8816 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:07.867127  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:07.874826  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:07.861068 14268 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.879551 14268 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.795365 14476 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:02:07.887070  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.889881  8743 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.891716 14460 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.889102 14268 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.863644 14408 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:07.861163 13827 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.900013  8748 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:07.819761 14412 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:07.864110 14475 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:07.894701 14464 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.001s
I20260709 13:02:07.875685 14405 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:07.905025 14402 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.006s
I20260709 13:02:07.912426  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.898242 14485 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:07.865573 14478 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.869717 14411 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.936941  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.942601  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.952557  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:07.954934  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.958240  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:07.958892  8816 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:07.961689  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:07.911652 14475 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.832304 14433 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:07.979167  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.915977 14405 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.914633 14420 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:07.978399  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.994117  8822 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:07.998507  8892 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:07.995502  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.014966  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2124) took 1127 ms. Trace:
I20260709 13:02:08.015605  8745 rpcz_store.cc:276] 0709 13:02:06.887382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.888143 (+   761us) service_pool.cc:224] Handling call
0709 13:02:06.902865 (+ 14722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.199128 (+296263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.014599 (+815471us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54000217d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:08.014877 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679950720}
I20260709 13:02:08.018805  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2445) took 1339 ms. Trace:
I20260709 13:02:08.019444  8889 rpcz_store.cc:276] 0709 13:02:06.679730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.004202 (+324472us) service_pool.cc:224] Handling call
0709 13:02:08.018554 (+1014352us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b540019e6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.018720 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142729472}
I20260709 13:02:07.943248 14402 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.025667  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:07.964334 14351 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:07.934974 14464 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.031396  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.037355  8889 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
W20260709 13:02:08.009344 14421 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 13:02:08.029794  8821 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:07.987659 14420 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:07.953508 14186 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:02:08.080984 14186 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:08.081576 14186 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:08.081236 14460 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.082604 14186 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.083331 14460 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.084175 14186 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:08.085876 14460 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.090452 14076 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:08.093197 14479 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:08.093927 14421 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.096647 14421 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.097009 14449 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.097873 14449 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:08.094766  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2659) took 1697 ms (client timeout 1882 ms). Trace:
I20260709 13:02:08.099727  8896 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:08.040313 14494 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 13:02:08.041875 14067 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:08.042795 14447 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:08.045940  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 949) took 2100 ms (client timeout 1767 ms). Trace:
W20260709 13:02:08.046649  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2655) took 1952 ms (client timeout 1913 ms). Trace:
I20260709 13:02:08.108213  8816 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:08.049883 14313 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.051019 14404 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:08.063524 14115 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.024912 13839 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:07.981593 14424 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:02:08.066233 14048 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:08.068594 14492 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:08.123997 14355 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:08.150048 14478 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.149812 14355 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.152041 14355 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:07.949223 13813 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.069952 14350 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.071128  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.071350 14411 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.019765 14398 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.019982 13368 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:08.072809 14485 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.075673 14402 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.076462  8895 raft_consensus.cc:2368] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:08.078119 14433 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.079648 14412 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:07.995184 14053 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.101346  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:08.183138  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81890.4us
I20260709 13:02:08.183813  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:08.102095  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2459) took 1090 ms. Trace:
I20260709 13:02:08.184993  8898 rpcz_store.cc:276] 0709 13:02:07.011375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.052748 (+ 41373us) service_pool.cc:224] Handling call
0709 13:02:08.101929 (+1049181us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540019ebd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.102041 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430720256}
W20260709 13:02:08.099679  8891 rpcz_store.cc:269] 0709 13:02:06.397179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.783511 (+386332us) service_pool.cc:224] Handling call
0709 13:02:08.094562 (+1311051us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b540046da54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.094708 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1677858688}
I20260709 13:02:08.009842  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:08.109037  8890 rpcz_store.cc:269] 0709 13:02:05.945028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.945159 (+   131us) service_pool.cc:224] Handling call
0709 13:02:08.045696 (+2100537us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.045829 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035104128}
W20260709 13:02:08.109813  8894 rpcz_store.cc:269] 0709 13:02:06.094717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.332556 (+237839us) service_pool.cc:224] Handling call
0709 13:02:08.046468 (+1713912us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000aafd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.046587 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998298880}
I20260709 13:02:08.122102  8824 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.127441  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.104369 14459 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.108786 14432 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.132053  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 312494us
I20260709 13:02:08.153355  8788 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:08.158962  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 955) took 2146 ms. Trace:
I20260709 13:02:08.199977  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.200205  8896 rpcz_store.cc:276] 0709 13:02:06.012386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.687476 (+1675090us) service_pool.cc:224] Handling call
0709 13:02:08.099524 (+412048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.100420 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.158672 (+ 58252us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b540046da50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.158876 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946291968}
I20260709 13:02:08.122164 14115 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:08.204466  8892 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.108795 14477 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:08.109120 14067 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:08.121904 14313 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.178197  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 107162us
I20260709 13:02:08.209054  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.149111 14048 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.180670 14455 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:08.201894  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 48638.3us
I20260709 13:02:08.211993  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.229486  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.123517 14404 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.191922  8889 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:08.182055 14445 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.214319 14497 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:08.217614 14115 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.215127  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:08.218847  8712 connection.cc:256] RPC call timeout handler was delayed by -1.3208359359999804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:02:08.219254 14425 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:02:08.221127 14399 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.005s
W20260709 13:02:08.225893 14502 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:08.225780  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.263108  8892 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:08.217208 14313 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.131927 14447 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:08.236155  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.195598 14412 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.164513 14492 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.276053  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:08.276636  8815 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.276654  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.167953 14350 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.207110 13709 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:08.207304 14460 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:08.241806  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.171345 14398 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.244059  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.290148  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.291098  8894 raft_consensus.cc:2410] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:08.236572 14391 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:08.241241 14033 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
W20260709 13:02:08.259639  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 47749.6us
I20260709 13:02:08.299595  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.228921 14455 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:08.267895  8890 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:08.238909 13368 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.269970  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.270124  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.264765 14115 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.274310 14313 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.274506  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:08.301362  8892 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:02:08.303139  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.207062 14432 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.202749 14459 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.295949  8815 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:08.287429 14351 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.325205  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.325910  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.326892  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.327862  9444 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:08.328751  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.265141 14478 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:08.229779 14477 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.303452  8824 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.243577 14449 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.208542 14420 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.288472 14460 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.350474  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.304528  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.323239  8821 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 11: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.330387  9441 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:08.333943  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.334621  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.327286  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:08.337488  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.303880  9449 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:08.342590  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.343119  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.305387  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.356033  9412 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:08.376892  8815 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:08.378114  8817 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:02:08.378276  9447 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:08.376974  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2483336459999919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 13:02:08.379487  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:08.379940  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.380664  8712 connection.cc:256] RPC call timeout handler was delayed by -1.2186336460000575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:02:08.381266  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1804359359999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
I20260709 13:02:08.380604  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:08.381661  8712 connection.cc:256] RPC call timeout handler was delayed by -1.1660359360000712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 13:02:08.351053  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 145125us
I20260709 13:02:08.351104  9443 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.385166  9442 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
W20260709 13:02:08.391525  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64370.6us
W20260709 13:02:08.392532  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.392590  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.393613  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.390883  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.406612  9444 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:08.297102 14405 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.409596  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.417094  8820 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 11.
I20260709 13:02:08.414757 13709 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:08.415496  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.417611  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:08.429425  8890 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:08.375993 14460 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.350946 14464 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.287731 14412 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.430321  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.437651  8890 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:08.438551  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.439548  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.440032  8890 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.431360  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:08.326208 13368 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.428622  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:08.395623 14420 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.409081 14313 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:08.449221  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 93018us
W20260709 13:02:08.457721  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.462332  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.457919 14405 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.379415 14449 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:02:08.432303 14510 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:08.460367 14412 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.455883  8892 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:08.403699 14507 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:02:08.428180 14433 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:08.462960  8894 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16. Re-sending same reply.
I20260709 13:02:08.425386 13839 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:08.331069 14067 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.476106  9443 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
W20260709 13:02:08.476966  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 177497us
I20260709 13:02:08.484454  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.499229  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:08.492542  8890 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:08.497021  9444 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:08.500792  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.501643  8890 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 10.
I20260709 13:02:08.502935  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.503976  8890 raft_consensus.cc:2410] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:08.431511 14402 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:02:08.446616 14355 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:08.432149 14475 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.498967  9441 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:08.507139  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.526655  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.529935  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.532508  8890 raft_consensus.cc:2410] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:08.474524 14313 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:08.481940 14405 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:08.537380  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.537079  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.579301  9442 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:08.580207  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 957) took 2479 ms. Trace:
I20260709 13:02:08.583837  8890 rpcz_store.cc:276] 0709 13:02:06.067982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.536600 (+2468618us) service_pool.cc:224] Handling call
0709 13:02:08.544247 (+  7647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.545366 (+  1119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.545620 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.545901 (+   281us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:08.547038 (+  1137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.547187 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.547382 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.475250 14510 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.537483  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.536978  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.588155  9445 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:08.537014  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.571972  8823 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.505334 14399 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.549744  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:08.573124  8862 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.132:38693 timed out after 1.508s (SENT) [suppressed 20 similar messages]
I20260709 13:02:08.591320  8894 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:08.597559  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.598431  8890 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:08.497710 14449 consensus_queue.cc:1243] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:02:08.601823  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.603758  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.465536 14455 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.550669 14513 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:08.592125 14504 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:02:08.629951  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:08.580528 14405 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.463568 14391 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.624485  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.506042 14463 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.099s
I20260709 13:02:08.546687 14404 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.630883  8894 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:08.635872  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.623006 14528 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.603675 14511 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:02:08.505787 13839 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.629652 14513 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:08.752048  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.689874 14492 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.619066 14425 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.677397  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.624133 14048 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.724623 14455 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.770897  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.673506 14350 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.668147 14398 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.594039 13709 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:08.672578 14516 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:08.674902 14405 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.619382 14449 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.605294 14399 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.690958 14475 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:08.623031 14515 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:08.694936  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:08.738281 14036 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:02:08.625598 14478 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:08.684156 14186 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.005s
W20260709 13:02:08.688176 14408 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.776s	user 0.005s	sys 0.005s
I20260709 13:02:08.694674 14485 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:02:08.680718 14525 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:08.695729 14353 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.675781 14498 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:08.689260  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:08.679108 13368 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.592578 14510 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:08.717007  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.794381 14492 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:08.587502 14412 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:08.719544  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2674) took 1005 ms. Trace:
I20260709 13:02:08.661625 14481 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.703387  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:08.727555  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.728515  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.836016  8740 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:08.730099  8749 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:08.743543  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.741822  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:08.750494  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:08.751299  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:08.751449  8741 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:08.838958  8744 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.752750  8742 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 11.
I20260709 13:02:08.755571  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.767563  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.572289 14532 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.692775  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:08.838709  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 135432us
I20260709 13:02:08.841632  9446 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.717538 14497 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:08.783838  8748 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:08.787447  8890 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
W20260709 13:02:08.791304  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.845201  8895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2457) took 1834 ms (client timeout 1822 ms). Trace:
I20260709 13:02:08.793519  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:08.796634  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101780us
I20260709 13:02:08.848181  8743 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:08.688504 14459 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.775063 14432 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:08.800483 14449 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.790462 14477 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:08.805552  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 232457us
I20260709 13:02:08.806046 14408 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.807293  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:08.791400 14527 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:08.793465 14040 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.820787 13368 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.888274 13368 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.718056  8896 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:02:08.805084 14268 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:08.752444 14528 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.774185 14425 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.756045 14313 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.826659  8892 rpcz_store.cc:276] 0709 13:02:07.713818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.303775 (+589957us) service_pool.cc:224] Handling call
0709 13:02:08.719314 (+415539us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.719469 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176133504}
I20260709 13:02:08.839042  8747 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.769186  8894 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:08.889434  8896 pending_rounds.cc:85] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:02:08.837116  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.895991  8896 pending_rounds.cc:107] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:02:08.821589 14353 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.844887 14534 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:08.818395 14498 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:08.799471  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.849931  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.850097  8890 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:02:08.906733  8890 pending_rounds.cc:85] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 8
I20260709 13:02:08.906975  8890 pending_rounds.cc:107] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:08.907248  8890 raft_consensus.cc:2889] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:08.850809  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:08.908126  8893 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:08.851521  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:08.847383  8895 rpcz_store.cc:269] 0709 13:02:07.010773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.271477 (+260704us) service_pool.cc:224] Handling call
0709 13:02:08.838410 (+1566933us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b540046ee54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:08.845123 (+  6713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848940416}
I20260709 13:02:08.866822  8741 pending_rounds.cc:85] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:08.852890 14539 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:08.866310  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:08.867633  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 140199us
I20260709 13:02:08.869361  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.866261  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.835458 14492 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:08.869809  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128097us
I20260709 13:02:08.806869  9443 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:08.889416  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:08.891305  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 162383us
I20260709 13:02:08.895511  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2657) took 2613 ms. Trace:
I20260709 13:02:08.913442  8894 rpcz_store.cc:276] 0709 13:02:06.281793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.741934 (+2460141us) service_pool.cc:224] Handling call
0709 13:02:08.768939 (+ 27005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.895251 (+126312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.895455 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.843043  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:08.896276  8896 raft_consensus.cc:2889] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:08.914749  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2656) took 2701 ms. Trace:
I20260709 13:02:08.915074  8896 rpcz_store.cc:276] 0709 13:02:06.213385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.717205 (+2503820us) service_pool.cc:224] Handling call
0709 13:02:08.717574 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.914513 (+196939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.914686 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.807505 14485 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:08.852210 14420 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:08.897759  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.903090  9444 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.922516  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.924374  8745 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:08.905511  8748 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:08.860188  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.909430  8741 pending_rounds.cc:107] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 14.11
W20260709 13:02:08.909446  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 58013.5us
W20260709 13:02:08.925171  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82231.5us
I20260709 13:02:08.909802  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.852419 14404 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.926155  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:08.925204  8741 raft_consensus.cc:2889] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:08.927685  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:08.892891 14268 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.929633  9448 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:08.923308 14040 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:08.930401 14498 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.892287 14432 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.929679  9446 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:08.932754 14528 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.837110 14412 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.887861 14527 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:08.968541 14268 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.975996 14478 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:08.848732 14481 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:08.934487  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:08.934828 14115 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.002s
I20260709 13:02:08.935514 14425 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.846572 14532 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:08.939698 14351 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.606s	user 0.005s	sys 0.005s
I20260709 13:02:08.940220  8743 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:08.941489  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:08.941630  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:08.944227 14536 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:02:08.923786 14447 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
I20260709 13:02:08.944324  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:09.012423  9441 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:08.946774 14353 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.947988  8744 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:08.947963  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:08.949580  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2176) took 1127 ms. Trace:
I20260709 13:02:08.952502 14411 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:08.954988  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.955250  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.957016  8749 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:08.957999 13839 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:08.958310  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2658) took 2576 ms. Trace:
I20260709 13:02:09.018587  8890 rpcz_store.cc:276] 0709 13:02:06.347408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.849479 (+2502071us) service_pool.cc:224] Handling call
0709 13:02:08.849763 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.923880 (+ 74117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.924073 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.990182 14498 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:09.019860  8890 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:08.959090 14445 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.715s	user 0.007s	sys 0.004s
I20260709 13:02:09.025990 14445 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.029421  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 960) took 2453 ms. Trace:
I20260709 13:02:09.017450 14411 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:09.029913  8890 rpcz_store.cc:276] 0709 13:02:06.567550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.019377 (+2451827us) service_pool.cc:224] Handling call
0709 13:02:09.019680 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.020244 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.020447 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.031472  8890 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:09.032138  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2663) took 2410 ms. Trace:
I20260709 13:02:09.032475  8890 rpcz_store.cc:276] 0709 13:02:06.621964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.030709 (+2408745us) service_pool.cc:224] Handling call
0709 13:02:09.031019 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.031919 (+   900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.032087 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:09.032837  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.819083 14513 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.033437 14412 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.033622  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.033924 14513 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.034893  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.035735  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.037197 14398 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:08.863209 14036 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:02:08.963474  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.038786 14492 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:09.039368  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 76004.1us
I20260709 13:02:08.966679  8747 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:09.040693 14518 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:09.040860 14420 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:08.962788  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.041024  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:09.041350 14518 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:09.041426  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:09.041697  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 79031.2us
I20260709 13:02:09.042309  8818 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:08.974529  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:09.042819  9446 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:09.042419 14518 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.043216  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:08.976861  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:08.966652 13827 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.058s	user 0.011s	sys 0.002s
I20260709 13:02:08.977895 14033 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:09.049933  8890 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:08.910665 14539 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.978418 14432 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.979099  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:08.988072  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53652.4us
I20260709 13:02:09.055480  8748 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:08.978732  9442 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:08.969836 14040 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:08.990983 14537 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:09.057659  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.968830 14404 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.993444 14481 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:08.960806 14533 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:08.995738 14497 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.040696 14391 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.065706 14404 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.038789 14036 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:08.999435 14053 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.073643  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:09.075459  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.002947  8892 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:02:08.967482 14534 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:09.004012  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91560.4us
I20260709 13:02:09.009177  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 956) took 2968 ms. Trace:
I20260709 13:02:09.094681  8893 rpcz_store.cc:276] 0709 13:02:06.034165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.707599 (+1673434us) service_pool.cc:224] Handling call
0709 13:02:07.707956 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.908507 (+1200551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.001981 (+ 93474us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003be054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:09.002251 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755725184}
I20260709 13:02:09.010977 14351 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:09.011106  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:09.100116  8743 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.017124  8816 raft_consensus.cc:2410] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:09.017951  8746 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:09.015358  8817 rpcz_store.cc:276] 0709 13:02:07.822322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.822519 (+   197us) service_pool.cc:224] Handling call
0709 13:02:07.822915 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:08.378598 (+555683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.949211 (+570613us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:08.949510 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276238592}
I20260709 13:02:09.022120  9449 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:09.030664 14411 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:08.971536  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:09.156097  9448 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:09.042430  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.044247  8740 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.046916  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:09.053215  8897 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
W20260709 13:02:09.157321  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110444us
I20260709 13:02:09.158313  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.056496  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2664) took 2418 ms. Trace:
I20260709 13:02:09.159245  8890 rpcz_store.cc:276] 0709 13:02:06.663380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.033172 (+2369792us) service_pool.cc:224] Handling call
0709 13:02:09.049775 (+ 16603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.050449 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.050970 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.055663  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:08.991804  8891 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:09.061161 14040 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:08.981345 14527 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.066679 14036 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:09.053577 14425 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.076485  8892 pending_rounds.cc:85] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 12
I20260709 13:02:09.098727  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.178372  8892 pending_rounds.cc:107] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:02:09.178779  8892 raft_consensus.cc:2889] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:09.055594 14420 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:09.152859  8893 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:09.054893 14539 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.970088 14485 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:09.153848  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48675.1us
I20260709 13:02:09.077538 13813 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.059438  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:09.162482  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2443) took 2482 ms. Trace:
W20260709 13:02:09.165532  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 123204us
I20260709 13:02:09.169078 14313 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:09.054636 14033 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:09.171758  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 116215us
W20260709 13:02:09.163647  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65331.6us
I20260709 13:02:09.176599  8740 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:09.183332  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2665) took 2505 ms. Trace:
W20260709 13:02:09.185331  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 86692us
I20260709 13:02:09.154352 14391 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.095451 14404 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:09.193915  9449 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:09.189654  8890 rpcz_store.cc:276] 0709 13:02:06.679079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.160265 (+2481186us) service_pool.cc:224] Handling call
0709 13:02:09.160663 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.160911 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.160948 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.161006 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:09.161051 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.161121 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.161254 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.196208  8817 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.196723  8816 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:09.193611  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.209625  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.210047  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2442) took 2531 ms. Trace:
I20260709 13:02:09.210274  8890 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:09.210451  8895 rpcz_store.cc:276] 0709 13:02:06.678751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.156333 (+2477582us) service_pool.cc:224] Handling call
0709 13:02:09.178116 (+ 21783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.197824 (+ 19708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.197861 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.197907 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:09.209821 (+ 11914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.209884 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.209982 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.211004  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.172406 13827 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.201395  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:09.170483 14533 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:09.202111  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.215561  8747 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:09.205196  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.206974  8889 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:09.200484  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2662) took 2633 ms. Trace:
I20260709 13:02:09.212483  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:09.216408  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.179036 14411 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:09.182509 14425 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.218987  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.182420 14537 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:09.214655  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:09.193625 14067 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.216729  8892 rpcz_store.cc:276] 0709 13:02:06.566661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.966013 (+2399352us) service_pool.cc:224] Handling call
0709 13:02:08.982579 (+ 16566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.198220 (+215641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.200427 (+  2207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.241997 14540 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.244840  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.245779  8892 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:09.247016  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.248862  8892 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:09.241142 14115 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:09.251056 14115 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:09.251564 14115 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:09.221740 14425 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.225984 14527 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.218314 14464 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:02:09.218446 14186 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:09.259378 14186 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.259790 14186 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:09.261240 14405 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.201848  8893 rpcz_store.cc:276] 0709 13:02:06.677671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.095884 (+2418213us) service_pool.cc:224] Handling call
0709 13:02:09.152412 (+ 56528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.183020 (+ 30608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.183268 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:09.263550 14519 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.605s	user 0.003s	sys 0.000s
I20260709 13:02:09.226547 14537 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.264650 14537 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.268784 14420 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:09.269908  8897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2668) took 1557 ms (client timeout 1535 ms). Trace:
I20260709 13:02:09.270308  9442 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:09.271426  9442 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:09.273155  9442 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.275432  9443 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:09.276734  9443 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:09.220894 14478 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:09.279320 14033 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.216424  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:09.220052  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.200763 13813 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:09.290130  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 70131.4us
I20260709 13:02:09.291322  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:09.218019  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 125555us
I20260709 13:02:09.295567 14432 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.296108 14350 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:09.296550 14350 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:09.296829 14445 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:09.297585 14445 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:09.298266 14547 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:09.224953  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:09.298995  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:09.189617 13839 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:09.308848  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.213464  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.241528  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.198628 14528 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.308184  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2446) took 2628 ms. Trace:
I20260709 13:02:09.310611  8892 rpcz_store.cc:276] 0709 13:02:06.680047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.248115 (+2568068us) service_pool.cc:224] Handling call
0709 13:02:09.248458 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.299559 (+ 51101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.308111 (+  8552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.319355  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.319957  9443 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.320200  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:09.168915 14510 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.296924 14350 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:09.270345  8897 rpcz_store.cc:269] 0709 13:02:07.712054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.862252 (+150198us) service_pool.cc:224] Handling call
0709 13:02:09.269741 (+1407489us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000488d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:09.269842 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2640700800}
I20260709 13:02:09.323853  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.222975  8818 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:09.300670  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.325014  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.263562  8893 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:09.311983  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98591.6us
I20260709 13:02:09.326051  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2447) took 2645 ms. Trace:
I20260709 13:02:09.326381  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.219260 14313 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:09.225911  8746 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:09.273255 14475 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:09.326417  8893 rpcz_store.cc:276] 0709 13:02:06.680389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.262916 (+2582527us) service_pool.cc:224] Handling call
0709 13:02:09.263335 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.325818 (+ 62483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.326003 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.329667  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2198) took 1053 ms. Trace:
I20260709 13:02:09.330204  8815 rpcz_store.cc:276] 0709 13:02:08.276298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.325672 (+ 49374us) service_pool.cc:224] Handling call
0709 13:02:09.329446 (+1003774us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b540001fc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:09.329593 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115669888}
W20260709 13:02:09.331390  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 89940.4us
I20260709 13:02:09.331862  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:09.347687  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.348380  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.366595  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.368448  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:09.322332 14555 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:09.313241 14447 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.369156  8823 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.309984 14405 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.313830 14464 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:09.219398 14391 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.372035  9443 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:09.375283  9447 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:09.376605  8893 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:09.324337 14472 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.378664  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2451) took 2596 ms. Trace:
I20260709 13:02:09.398454  8893 rpcz_store.cc:276] 0709 13:02:06.782478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.329597 (+2547119us) service_pool.cc:224] Handling call
0709 13:02:09.376305 (+ 46708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.378406 (+  2101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.378603 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.380000 14313 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.381747 14558 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:09.383072 14534 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.368484 14412 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:02:09.384361  9443 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:09.329303 14475 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:09.388608 14040 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.298620 14445 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.310761 14404 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:02:09.393699  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.394042 14115 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.394234 14539 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.397789  8895 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:02:09.385324 14551 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:02:09.372428 14460 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.941s	user 0.008s	sys 0.004s
I20260709 13:02:09.372430 14497 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:09.318774 14268 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.402920 14529 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:09.378679 14521 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:09.428238  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 968) took 1261 ms. Trace:
I20260709 13:02:09.432937  8893 rpcz_store.cc:276] 0709 13:02:08.166829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.408555 (+1241726us) service_pool.cc:224] Handling call
0709 13:02:09.427583 (+ 19028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.427819 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.427856 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.427910 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:09.427953 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.428023 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.428170 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.415367  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2450) took 2734 ms. Trace:
I20260709 13:02:09.399611 14464 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:09.369257 14540 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 13:02:09.438747  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 233620us
I20260709 13:02:09.317521 14528 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.432031  9448 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:09.368170 14481 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:02:09.436002  8895 rpcz_store.cc:276] 0709 13:02:06.719244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.365205 (+2645961us) service_pool.cc:224] Handling call
0709 13:02:09.377566 (+ 12361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.415038 (+ 37472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.415293 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.328722 14036 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.472908  8893 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:02:09.473557  8893 pending_rounds.cc:85] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 6
I20260709 13:02:09.473894  8893 pending_rounds.cc:107] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:09.474236  8893 raft_consensus.cc:2889] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:09.475131  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2679) took 1054 ms. Trace:
I20260709 13:02:09.475600  8893 rpcz_store.cc:276] 0709 13:02:08.420652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.436984 (+1016332us) service_pool.cc:224] Handling call
0709 13:02:09.472351 (+ 35367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.474816 (+  2465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.475062 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.431807 14412 consensus_queue.cc:1243] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:09.414940 14533 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:09.437731 14464 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.435374 14497 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:09.502115  8893 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:09.438897 14534 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:09.503688  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 171974us
I20260709 13:02:09.504441  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.408023 14472 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.466605 14447 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.505607  8895 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:09.506574  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66917.3us
I20260709 13:02:09.430032 14460 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.478350 14521 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.499239 14412 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:09.526283  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 287653us
I20260709 13:02:09.530604  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.522367  8823 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:09.505439 14534 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:09.508394  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.539146  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.518642 14521 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:09.542465  8897 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:09.543061  8743 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:09.544155  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:09.544898  8742 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:02:09.545238  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 325918us
I20260709 13:02:09.525141 14036 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.524423 14412 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:09.549121  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:09.547449  8895 raft_consensus.cc:1217] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Deduplicated request from leader. Original: 14.13->[17.14-17.14]   Dedup: 17.14->[]
I20260709 13:02:09.479174 14475 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.518873 14447 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.555089  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:09.546392  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.556641  8895 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:02:09.551759 14498 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:09.570281  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.571794  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.573434  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.544296 14521 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.547633 14570 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:09.594607 14561 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:02:09.595330 14560 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:02:09.578299 13839 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.597800 13839 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.598018 14412 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.599531 14412 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.599972 13839 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.554388 14523 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:02:09.602878 14399 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.603472 14399 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:09.603838 14399 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.604158 14399 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.605437 14476 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.605968 14476 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.607766 14425 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.608279 14476 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:09.609622 14539 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.630306  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2194) took 1374 ms. Trace:
I20260709 13:02:09.632444  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:09.589416 14475 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.633821  8823 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:02:09.637423  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 243880us
I20260709 13:02:09.569516 14571 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:09.638183  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:09.590952 14391 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.642926  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:09.639703  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:09.593904 14268 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.645712  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.569594 14534 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.648120 14534 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.598019 14472 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:09.634312  8821 rpcz_store.cc:276] 0709 13:02:08.255448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.436601 (+181153us) service_pool.cc:224] Handling call
0709 13:02:08.462387 (+ 25786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.790845 (+328458us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:08.909095 (+118250us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.309613 (+400518us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.310231 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.310383 (+   152us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:09.312350 (+  1967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.312440 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:09.630026 (+317586us) raft_consensus.cc:1685] finished
0709 13:02:09.630049 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.630234 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65656,"prepare.queue_time_us":25592,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":138517,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":23981,"thread_start_us":756370,"threads_started":3,"wal-append.queue_time_us":448779}
I20260709 13:02:09.651266  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:09.654136 14566 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:02:09.655396 14563 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:09.657189 14579 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:09.636546 14040 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:09.659066 14447 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:09.659828 14420 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:09.662432  9442 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:09.664165 14578 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.665768 14460 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:09.666225 14568 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:02:09.674345 14411 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.557717 14497 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:09.668989  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 119446us
I20260709 13:02:09.638458 14498 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.681206  8821 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:02:09.705262  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.682607  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.706862  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.642433 14571 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.650159 14539 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:09.649660 14391 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.739507  8789 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:09.656790 14521 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.650424 14115 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.690685 14579 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:09.714973 14539 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:09.689224 14528 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.718029 14571 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.724911 14498 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.708719 14411 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.732642  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:09.735685  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:09.736209  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:09.737118  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:09.718375 14564 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:09.828343 14588 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:09.691474 14574 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.690056 13827 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:09.743070 14447 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
W20260709 13:02:09.743803 14566 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:02:09.747429 14186 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.749594  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:09.846484  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:09.846771 14498 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:09.834460 14464 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.691303 14420 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.849184 14420 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:09.752662  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.754249  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:09.851006  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 96817.6us
W20260709 13:02:09.851917  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.852775  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.700899 14510 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.763796  9442 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:09.858904  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1129) took 1177 ms. Trace:
I20260709 13:02:09.739889 14460 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:09.811156  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.825398  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.826261  8747 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:09.800959  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:09.826092  8893 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:09.829865  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 92852.1us
I20260709 13:02:09.861794  8748 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:02:09.862093  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:09.863826  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:09.824926 14432 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.841840 14067 consensus_meta.cc:287] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.004s
I20260709 13:02:09.811769 14579 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.833016 14574 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:09.743144 14350 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.869553  8822 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:02:09.870548  8822 pending_rounds.cc:85] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 12
I20260709 13:02:09.870824  8822 pending_rounds.cc:107] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:02:09.871152  8822 raft_consensus.cc:2889] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:09.842615 14521 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.769860 14539 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:09.762583 14540 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:09.759603 14425 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.865238  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:09.826447 14391 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.859434  9442 rpcz_store.cc:276] 0709 13:02:08.681872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.269627 (+587755us) service_pool.cc:224] Handling call
0709 13:02:09.270079 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.662172 (+392093us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b58009704d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:02:09.858521 (+196349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.858823 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78432640}
I20260709 13:02:09.866793 14584 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:09.867849 14564 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.869547 14574 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:09.834913 14472 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.834131 14496 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.770087 14481 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 13:02:09.862200 14455 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:09.870606 14528 raft_consensus.cc:2764] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:02:09.877388 14507 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:02:09.893510  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.877079 14540 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.895124 13368 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:09.895933  8747 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:09.894224 14521 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:09.894347 14186 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:09.762918 14537 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:09.916079  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.919308 14350 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:09.902191 14455 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:09.922617 14455 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:09.925470  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.890336 14432 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.897382 14472 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:09.909811  8893 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:09.893337 14584 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.900447 14540 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.892716 14391 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:09.878476 14587 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:02:09.988101  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:09.989053  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.958067 14472 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.908782 14521 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:09.988786 14540 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:09.898703 14496 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.038097  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2398) took 1249 ms. Trace:
I20260709 13:02:09.916823 14420 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.042646  8823 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:10.044251  8749 rpcz_store.cc:276] 0709 13:02:08.789469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.844489 (+ 55020us) service_pool.cc:224] Handling call
0709 13:02:08.844802 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.016682 (+171880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.037870 (+1021188us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:10.038038 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257652480}
I20260709 13:02:10.058625  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.058152  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.059372  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:09.998476 14460 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:10.064625  8893 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:10.062157  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.057474 14353 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 13:02:10.051798 14569 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.322s	user 0.005s	sys 0.007s
W20260709 13:02:10.061021 14590 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.152s	user 0.008s	sys 0.003s
I20260709 13:02:10.042006 14521 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:10.067211  9442 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:10.066260  8746 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:10.071219  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.075008  8749 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:10.062892 14186 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.075917  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.081166  8746 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:10.081481  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.079178 14408 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.083863  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.082851  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.066659 14411 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:10.078843 14574 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.084883  8743 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:02:10.060853 14399 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.092496  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.093221  8893 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:10.093545  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.087358  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.094141  8743 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:10.094995  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.095719  8743 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:10.074502 14391 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.099606  8746 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:10.100420  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.101349  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.101789  8746 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.101976  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.105173  8749 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:10.107873  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.109088  8743 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:02:10.142457  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.170822  8740 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:10.211658  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.216216  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.219738  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.220548  8743 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:02:10.230666  8712 connection.cc:256] RPC call timeout handler was delayed by -1.9429379220000556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:02:10.231333  8712 connection.cc:256] RPC call timeout handler was delayed by -1.761137921999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 13:02:10.231738  8712 connection.cc:256] RPC call timeout handler was delayed by -1.627937922000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:02:10.232108  8712 connection.cc:256] RPC call timeout handler was delayed by -1.4236379220000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
I20260709 13:02:10.253667  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.083402 14186 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.255759  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.085800 14539 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.261715  8743 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:10.263301  8743 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:10.213228  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.264547  8743 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:10.267868  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.305033  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.304623 14391 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:10.306381 14391 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:10.306512  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:10.309484 14537 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.309621 14481 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.310096 14481 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.259033 14579 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:10.272500 14595 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:02:10.279913 14593 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:10.270550 14587 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.302214 14540 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.314261 14481 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.334197  8746 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:10.335750  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.336311  8894 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:10.329090 14420 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:10.316493 14584 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.321662 14432 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.346165  8746 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:02:10.330973 14496 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:10.335428 14534 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.006s
I20260709 13:02:10.348677  8746 pending_rounds.cc:85] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 13
I20260709 13:02:10.349697  8746 pending_rounds.cc:107] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:02:10.350306  8746 raft_consensus.cc:2889] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:10.360602  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.359809 14507 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.355330 14584 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:10.362269 14507 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.364689 14507 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.366022 14186 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:10.370224  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:10.370609  8712 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.535s (SENT)
I20260709 13:02:10.371117  8894 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:10.371147  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:10.373879 14399 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:10.374562 14591 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.530s	user 0.001s	sys 0.000s
I20260709 13:02:10.374915 14420 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:10.375934  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2157) took 1643 ms. Trace:
I20260709 13:02:10.376289  8745 rpcz_store.cc:276] 0709 13:02:08.732932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.733399 (+   467us) service_pool.cc:224] Handling call
0709 13:02:10.375752 (+1642353us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b54000221d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.375858 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194086656}
I20260709 13:02:10.377748  8744 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:10.374042  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.390285 14529 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.390970 14529 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:10.391817  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 834296us
I20260709 13:02:10.394198  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:10.395888  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.396507  8894 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.397025  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.399235  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:10.401156  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.386099 14534 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:10.415699  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.417711  8897 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:10.401467 14357 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:10.419516  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:10.421370  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.422307  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80799.1us
I20260709 13:02:10.423280  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.425122  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.425801  8897 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:10.427047  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:10.432711  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.433331  8893 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.438027  8746 raft_consensus.cc:1217] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[14.9-17.11]   Dedup: 14.9->[15.10-17.11]
I20260709 13:02:10.438832  8746 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [14.9-17.11]
I20260709 13:02:10.409685 14432 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:02:10.412745 14411 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:02:10.440585  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:10.411916 14408 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:02:10.448299  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.450146  9441 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:10.437529 14497 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:02:10.456499  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.457803  8893 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:10.458209  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:10.458854  9441 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:10.459237  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.459511  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.460237  8746 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:10.461767  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.462412  8746 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:10.462445  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:10.446162 14455 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:02:10.439893 14537 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:02:10.469763  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.471433 14408 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.471887 14408 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.472216 14408 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.473131  8746 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:10.474633  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:10.461195  8893 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:02:10.478840  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75762.1us
I20260709 13:02:10.480484  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.481084  8895 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.437803 14350 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:10.483158  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.469096 14472 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.482461 14485 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:10.491624  8893 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:10.492736  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.483355 14564 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:10.494282 14564 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.496850 14564 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:10.497774 14571 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.002s
I20260709 13:02:10.497969 14564 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:10.498236 14571 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.499202  8893 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:10.493784  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.483753 14432 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:10.507244  8746 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:02:10.514676  8891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2675) took 2787 ms (client timeout 1885 ms). Trace:
W20260709 13:02:10.515165  8891 rpcz_store.cc:269] 0709 13:02:07.726940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.186361 (+459421us) service_pool.cc:224] Handling call
0709 13:02:10.514491 (+2328130us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.514601 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725190016}
I20260709 13:02:10.514740  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.516382  8748 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.519862  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.521917  8743 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:10.519236 14481 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.525520 14587 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:10.527518 14481 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.539068 14507 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:10.539549 14507 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:10.540431 14507 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.541749 14507 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:10.543999  8895 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:10.544536 14521 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:10.536392  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.527675 13813 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:10.550534 14537 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.551685 14537 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.552932 14472 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.553828 14537 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.515345 14571 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.562464  8896 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:10.580158  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.580943  8896 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:10.550998  8894 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:10.569847  8749 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:10.549875  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:10.582780  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.583824  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.583704  8891 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:10.545018  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.586477  8895 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:10.587172  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 105648us
W20260709 13:02:10.562762 14599 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.203s	user 0.000s	sys 0.004s
I20260709 13:02:10.585922 14455 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.586405 14432 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:10.555984 14408 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.589102 14455 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:10.589784 14432 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.588605 14350 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:02:10.597500  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.591328 14455 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.597832  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.598044  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.600426  8891 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17.
I20260709 13:02:10.564410 14529 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.582654 14496 raft_consensus.cc:2764] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:10.581117  8748 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:10.591145 14408 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.605262  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.621574  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:10.622354  8896 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:10.607126  8895 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:10.609863  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:10.623553  8821 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:10.612736  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.624866  8894 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:10.626124  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.626308  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.626411  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.627027  8895 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:10.615212  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.584676 14497 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:10.649499  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2476) took 2104 ms (client timeout 2368 ms). Trace:
I20260709 13:02:10.649617 14497 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:10.650228  8889 rpcz_store.cc:269] 0709 13:02:08.545502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.688911 (+143409us) service_pool.cc:224] Handling call
0709 13:02:10.649254 (+1960343us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b540015d954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.649423 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780125568}
I20260709 13:02:10.651755  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.627666  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.658367  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.660099 14521 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.660279  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:10.622675 14425 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:10.634974  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2778) took 1438 ms. Trace:
I20260709 13:02:10.622809 14472 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:10.661530  8819 rpcz_store.cc:276] 0709 13:02:09.196326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.196937 (+   611us) service_pool.cc:224] Handling call
0709 13:02:09.218378 (+ 21441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.242696 (+ 24318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.242727 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.242771 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:09.242791 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.634656 (+1391865us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b54003e35d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:10.634727 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.634893 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15888128}
I20260709 13:02:10.654520  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 969) took 1003 ms. Trace:
I20260709 13:02:10.664000  8823 rpcz_store.cc:276] 0709 13:02:09.650872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.651014 (+   142us) service_pool.cc:224] Handling call
0709 13:02:10.654315 (+1003301us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400083ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.654459 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857278208}
I20260709 13:02:10.664618 14513 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:10.665462 14412 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:10.665814 14513 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:10.666425 14513 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:10.666139 14412 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.619386 14557 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:02:10.657618  8889 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:10.643043  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:10.669970  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2406) took 1730 ms (client timeout 2096 ms). Trace:
I20260709 13:02:10.670382  8897 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:10.670509  8742 rpcz_store.cc:269] 0709 13:02:08.939148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.941377 (+  2229us) service_pool.cc:224] Handling call
0709 13:02:10.669759 (+1728382us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001d9054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.669908 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235458944}
I20260709 13:02:10.671890  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:10.646170  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:10.647308  9447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 976) took 2139 ms (client timeout 1875 ms). Trace:
W20260709 13:02:10.669973  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.672461  8742 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.674966  9442 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.677421  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:10.652264 14497 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.654506  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2412) took 1713 ms. Trace:
I20260709 13:02:10.618613  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:10.688730  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70252.6us
I20260709 13:02:10.655468  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.658434  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:10.690603  8891 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:10.631220 14353 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:10.675649  9447 rpcz_store.cc:269] 0709 13:02:08.507622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.508863 (+  1241us) service_pool.cc:224] Handling call
0709 13:02:10.646992 (+2138129us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400499b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.647183 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1810281216}
I20260709 13:02:10.687391  8744 rpcz_store.cc:276] 0709 13:02:08.940756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.014378 (+ 73622us) service_pool.cc:224] Handling call
0709 13:02:10.377400 (+1363022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.378090 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.654231 (+276141us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000221d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:10.654422 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3133941632}
I20260709 13:02:10.636601  8895 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:10.652880  9444 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 17. Ops: []
I20260709 13:02:10.681203  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.703816  8892 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:10.661824 14186 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:10.681814 14408 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:10.684549 14574 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:10.661926 14425 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:02:10.686192  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66123.2us
I20260709 13:02:10.686542  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 966) took 1317 ms. Trace:
I20260709 13:02:10.693739  9441 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:10.705201  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.710414  8815 rpcz_store.cc:276] 0709 13:02:09.369359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.369470 (+   111us) service_pool.cc:224] Handling call
0709 13:02:09.369802 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.379095 (+  9293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.379138 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.379191 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:09.379232 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.686286 (+1307054us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b54000839d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:10.686332 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.686487 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20928768}
I20260709 13:02:10.711558  8895 raft_consensus.cc:2376] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17. Re-sending same reply.
I20260709 13:02:10.708459  8897 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:10.712821  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.714946  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:10.724586  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:10.733479  8891 raft_consensus.cc:2368] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16.
I20260709 13:02:10.725750  8892 raft_consensus.cc:2393] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:10.727878  9442 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:10.723927  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:10.742022  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:10.745689  8742 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:10.750571  8712 leader_election.cc:400] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17.
I20260709 13:02:10.706323 14578 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.752573  8712 leader_election.cc:403] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:10.707080 14357 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:10.747709  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:10.737560 14432 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.706941 14186 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:10.715217 14425 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:10.667546 14513 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:10.706528 14520 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:02:10.753961  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.019372  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:11.019761  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 272205us
W20260709 13:02:10.799150  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.922122  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.020871  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:10.861300 14553 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.641s	user 0.005s	sys 0.000s
W20260709 13:02:11.021776  9449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 975) took 2514 ms (client timeout 1810 ms). Trace:
I20260709 13:02:11.022037 14529 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:11.022214  9449 rpcz_store.cc:269] 0709 13:02:08.507334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.507438 (+   104us) service_pool.cc:224] Handling call
0709 13:02:11.021568 (+2514130us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54001937d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:11.021702 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995872128}
I20260709 13:02:11.022749 14186 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.023450  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.023509 14513 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:10.864060  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1118) took 2263 ms. Trace:
I20260709 13:02:11.024655  9448 rpcz_store.cc:276] 0709 13:02:08.604056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.172926 (+568870us) service_pool.cc:224] Handling call
0709 13:02:09.173138 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.440170 (+267032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.830501 (+1390331us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54006a95d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:10.863973 (+ 33472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559196544}
I20260709 13:02:11.025581  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.025882  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.026154  8890 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.721556 14507 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:10.860317  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:10.753523 14602 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:10.754295 14574 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:10.824934 14464 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:10.753263 14350 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.019680  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:10.822827  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:10.916684  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:10.806097  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.027325  9445 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:11.023554  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2400) took 2220 ms. Trace:
I20260709 13:02:10.861415  8815 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:11.020759  9412 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:11.021090  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 362757us
I20260709 13:02:11.046693  8787 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.046792  8742 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:11.026966  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.024796  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.048516  9449 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.806468  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.048615  9448 raft_consensus.cc:2410] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:11.023959 14186 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:10.921581  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.020963  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.025765 14432 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.050084  8748 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.057564  8891 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:11.053344  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:10.740041 14353 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.026784 14507 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.721923 14497 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:11.052338  8817 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:11.065828  8815 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:11.066252  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45598.8us
I20260709 13:02:11.075726 14353 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.074795 14602 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:11.076154  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:11.077447 14481 consensus_queue.cc:1059] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:11.077972 14353 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.078567  9440 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:11.078644 14481 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:11.079190 14481 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.079720 14481 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.080531 14432 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:11.043292  8741 rpcz_store.cc:276] 0709 13:02:08.803840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.845877 (+ 42037us) service_pool.cc:224] Handling call
0709 13:02:08.866614 (+ 20737us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:08.975747 (+109133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.975784 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:08.980516 (+  4732us) log.cc:844] Serialized 78 byte log entry
0709 13:02:09.028890 (+ 48374us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:09.052278 (+ 23388us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.054266 (+  1988us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.056601 (+  2335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.056700 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.829341 (+1772641us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:11.023333 (+193992us) raft_consensus.cc:1685] finished
0709 13:02:11.023355 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.023482 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81411,"prepare.queue_time_us":879,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":90,"raft.queue_time_us":24268,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":208,"spinlock_wait_cycles":38325504,"thread_start_us":38731,"threads_started":1,"wal-append.queue_time_us":174224}
I20260709 13:02:11.079224 14507 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.077597  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.081950  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.082543  8741 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.085593  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.086063 14464 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:11.086599 14464 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:11.086838  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.087410  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.088212  8824 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:11.072295  9446 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:11.082139  8748 raft_consensus.cc:2368] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:02:11.090773 14540 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.090469 14464 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.091089  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.091534 14616 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.091846  8748 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:11.092079 14616 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.092803  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.074128 14607 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:11.094223 14616 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.093973 14607 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.095952 14353 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.096218 14607 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.013896 14408 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.096441 14353 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.096993 14408 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.099261 14408 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.100064  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.100740  8742 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:11.100564 14481 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.101460 14616 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:11.091172 14540 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:10.722934 14611 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:11.103317  8741 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:11.102586 14540 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.724870 14533 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.348s
I20260709 13:02:11.104323  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.105154  8741 raft_consensus.cc:2410] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:11.051932 14587 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:11.103147 14616 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.101831  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.106247  8748 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:10.887864 14518 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.106796  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.106040  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.065052 14571 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.107708 14571 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.109645  8746 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:11.109756 14571 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:10.741117 14485 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:11.107908  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.112907 14485 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:11.103093 14611 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:11.107038  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.052279 14606 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.115131 14611 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.115314 14606 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.116343  8742 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:11.117583 14606 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:11.118285  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.119046  8897 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.119943  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.120464  8897 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:11.117594  8742 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:11.114351 14485 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.115690 14353 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.124627  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.128640  8815 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:11.121218  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.134696  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.135046  8889 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:02:11.135196  8897 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.135516  8815 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.119059  8748 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 12: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.140625  8890 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:11.142345  8741 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.135475  8889 pending_rounds.cc:85] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 10
I20260709 13:02:11.143361  8889 pending_rounds.cc:107] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:02:11.143716  8889 raft_consensus.cc:2889] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:11.144311  8890 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:11.145157  8889 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:11.142903  8815 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:11.148461  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.149115  8815 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:11.150278  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.150435  8748 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:11.150940  8815 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.151880  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.153096  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:11.159333  9449 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:10.861310 14610 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:11.141649  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:11.160697  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 410329us
I20260709 13:02:11.153000  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.167116  8815 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
W20260709 13:02:11.167426  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.172056  8891 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:11.174662  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.175454  8824 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.124650 14616 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.142735 14604 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.107170 14481 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.196679  8892 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:11.142354 14520 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17.
I20260709 13:02:11.178229  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.258901 14412 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.020753 14578 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:11.257150  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 2.28916e+06us
I20260709 13:02:11.107111 14518 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:11.264243  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 188167us
I20260709 13:02:11.271059  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:11.125407 14507 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.259652  8824 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:11.146437 14353 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.250345 14529 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:11.173384 14610 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.274691  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:11.258610 14481 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:11.272238  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.263716 14602 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.293654 14580 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.294169 14580 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:11.294566 14580 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.295783  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:11.281953  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.282330  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.300733  8891 raft_consensus.cc:2368] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14.
I20260709 13:02:11.301808  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.295159 14602 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.302403  8891 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.302975  8892 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.303558 14580 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.306258  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:11.307288  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:11.320165  8820 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:11.279419 14616 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.327980  8862 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:11.246735 14425 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.267870 14540 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.292490 14481 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.239812 14604 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.279834 14412 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:02:11.334354 14464 raft_consensus.cc:2764] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:11.281740 14353 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:11.472338  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 319293us
I20260709 13:02:11.330193  8891 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:11.313767 14537 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:11.421131  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.386508 14520 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.475071  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.319242 14529 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.479423  8815 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:11.484136  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.417868 14425 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.482321  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.488202  8824 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
W20260709 13:02:11.489010 14618 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:02:11.489800 14620 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:02:11.346733 14357 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:11.493054 14614 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.776s	user 0.002s	sys 0.000s
I20260709 13:02:11.494208  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2171) took 1061 ms. Trace:
I20260709 13:02:11.494356 14353 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.483820  8748 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:11.551941 14357 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.552965  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.553611  8748 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.554425 14518 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.475836 14623 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:11.512036  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.513988 14617 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.314s	user 0.000s	sys 0.000s
W20260709 13:02:11.519500  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.523075  8893 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.369728 14408 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:11.458076 14481 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:11.541496  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.611292  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.333734 14411 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 13:02:11.562671  8748 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:11.566496  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.612632  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.583830 14580 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.589730 14449 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.591452  8889 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:11.613745  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:11.594151 14350 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.003s
I20260709 13:02:11.595369  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1113) took 3052 ms. Trace:
W20260709 13:02:11.591184  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2676) took 3807 ms (client timeout 1737 ms). Trace:
I20260709 13:02:11.596041 14532 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:11.597398  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 428525us
I20260709 13:02:11.597627  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:11.327625  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.552006  8746 rpcz_store.cc:276] 0709 13:02:10.432386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.110631 (+678245us) service_pool.cc:224] Handling call
0709 13:02:11.110950 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.164147 (+ 53197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.164183 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.199249 (+ 35066us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.484995 (+285746us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:11.485047 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.485133 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.494008 (+  8875us) raft_consensus.cc:1685] finished
0709 13:02:11.494028 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.494148 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39901,"raft.queue_time_us":53212,"spinlock_wait_cycles":7306496,"thread_start_us":207283,"threads_started":1,"wal-append.queue_time_us":286683}
I20260709 13:02:11.371469 14571 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.166s
I20260709 13:02:11.475713 14622 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:11.612815  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.614494  8748 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:11.635653  8891 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:11.641870  8892 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:11.644781  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.645521  9449 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:11.620525  8898 rpcz_store.cc:269] 0709 13:02:07.784187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.198517 (+414330us) service_pool.cc:224] Handling call
0709 13:02:11.590783 (+3392266us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b5400249354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:11.591037 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892789504}
I20260709 13:02:11.617121  9444 rpcz_store.cc:276] 0709 13:02:08.543268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.105746 (+562478us) service_pool.cc:224] Handling call
0709 13:02:10.652589 (+1546843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.703073 (+ 50484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.595053 (+891980us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400499b50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:11.595253 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3420341504}
W20260709 13:02:11.634150  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 338435us
I20260709 13:02:11.672791  8893 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:11.682024  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:11.684726  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.685393  8893 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.651141  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:11.687558  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:11.687985  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.688320  8892 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.684262  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1142) took 1077 ms. Trace:
I20260709 13:02:11.694134  8893 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
W20260709 13:02:11.694290  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 368945us
I20260709 13:02:11.652570  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:11.695369  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.695446  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:11.616267 14623 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.701267  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.703266  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:11.638077 14618 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:11.520429 14520 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.704645  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:11.691893  9448 rpcz_store.cc:276] 0709 13:02:10.606582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.068437 (+461855us) service_pool.cc:224] Handling call
0709 13:02:11.068874 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.069450 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.069491 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.070970 (+  1479us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.178248 (+107278us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:11.178306 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.178399 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.684058 (+505659us) raft_consensus.cc:1685] finished
0709 13:02:11.684078 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.684203 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":34444,"spinlock_wait_cycles":1052544,"thread_start_us":106456,"threads_started":1,"wal-append.queue_time_us":198607}
I20260709 13:02:11.623288 14408 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:11.709848  8748 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:02:11.710047  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:11.716610  8892 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:11.646204 14602 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:11.646049 14350 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:11.726601  8893 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:11.682852 14518 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.731686  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.772146  8891 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:11.697072 14615 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:11.700064 14472 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.771030  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2499) took 1157 ms. Trace:
I20260709 13:02:11.768354 14611 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:11.801666 14611 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:11.800257  8891 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:11.802956 14611 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.772046 14602 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.784358 14518 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.805078 14602 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.807665 14518 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:11.808470  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.804955  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 987) took 1111 ms. Trace:
I20260709 13:02:11.810926  8891 rpcz_store.cc:276] 0709 13:02:10.693274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.772835 (+1079561us) service_pool.cc:224] Handling call
0709 13:02:11.798187 (+ 25352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.803147 (+  4960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.804536 (+  1389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.812791  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2691) took 1224 ms. Trace:
I20260709 13:02:11.785265 14411 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.813205  8889 rpcz_store.cc:276] 0709 13:02:10.588105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.162598 (+574493us) service_pool.cc:224] Handling call
0709 13:02:11.590970 (+428372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.613785 (+ 22815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.812586 (+198801us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400249350. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:11.812731 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984412544}
I20260709 13:02:11.813616 14411 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.814908  8889 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:11.816174 14537 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:11.785781 14472 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.785817  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 975) took 1873 ms. Trace:
I20260709 13:02:11.814256  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.787159  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 974) took 1876 ms. Trace:
I20260709 13:02:11.819319  8749 rpcz_store.cc:276] 0709 13:02:09.910588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.274232 (+363644us) service_pool.cc:224] Handling call
0709 13:02:10.274572 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.583168 (+308596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.786877 (+1203709us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54004673d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:11.787094 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678539136}
W20260709 13:02:11.788829  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.789156  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 996) took 1308 ms. Trace:
I20260709 13:02:11.790120  8893 rpcz_store.cc:276] 0709 13:02:10.614273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.695372 (+1081099us) service_pool.cc:224] Handling call
0709 13:02:11.725240 (+ 29868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.770315 (+ 45075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.770965 (+   650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.768729 14507 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.788586 14455 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:11.788784 14631 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:11.795313 14186 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.000s
I20260709 13:02:11.818374  8743 rpcz_store.cc:276] 0709 13:02:09.912763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.305936 (+393173us) service_pool.cc:224] Handling call
0709 13:02:10.306235 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.522345 (+216110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.785491 (+1263146us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000b4d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:11.785754 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495604992}
I20260709 13:02:11.816469  8889 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:11.701447 14449 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:02:11.831662  8746 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:11.869370  8819 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:11.834233  8742 rpcz_store.cc:276] 0709 13:02:10.480729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.122088 (+641359us) service_pool.cc:224] Handling call
0709 13:02:11.122386 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.164293 (+ 41907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.164341 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.164890 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.385811 (+220921us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:11.385872 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.385954 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.788940 (+402986us) raft_consensus.cc:1685] finished
0709 13:02:11.788963 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.789098 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41486,"raft.queue_time_us":48144,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":23730,"thread_start_us":170000,"threads_started":1,"wal-append.queue_time_us":431807}
W20260709 13:02:11.836930  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 141599us
I20260709 13:02:11.826828 14472 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:11.870812  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 175530us
I20260709 13:02:11.873005  8893 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:11.718715 14485 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:02:11.843302  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.876518  8824 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:11.805648 14579 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.718400 14529 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.875526  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.838677 14578 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.842942 14455 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:11.863107 14610 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.847491 14571 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:11.907061  9448 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:11.899608  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:11.865950 14622 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.903579  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.904174  9446 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:11.905915  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:11.915316  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.916056  9448 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:11.916741  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.907917  8893 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:11.870817 14532 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.911422  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.918432  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.918912  9448 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:11.920212  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.920567  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:11.913154  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.893543 14643 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:11.925839 14455 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:11.900324 14507 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.927907 14520 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:11.912940 14579 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:11.919862 14578 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:11.929888  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:11.899156 14631 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:11.907682 14616 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.003s
I20260709 13:02:11.933786  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:11.897574 14411 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.936702  8893 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:11.942214 14644 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:11.934926 14580 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:11.943210 14650 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:11.879523 14623 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:11.953929 14633 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:02:11.914541 14571 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:11.963445  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 124630us
I20260709 13:02:11.977537  9441 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
W20260709 13:02:11.980803  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47149.7us
I20260709 13:02:11.874329 14408 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:11.931669 14532 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:11.990382  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 117623us
I20260709 13:02:12.004132  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.019023  8815 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.006626  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.987323 14412 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.026405  8817 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:12.023519  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.987208 14616 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.029222  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.032256  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:11.977483 14507 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.000922 14580 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:12.041775  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:11.938046 14643 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.044152  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.044636  8815 raft_consensus.cc:2368] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:02:12.025328 14650 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.021986 14459 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.044879  8818 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:12.029783 14604 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:11.987233 14631 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.027264 14628 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:02:12.080649  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2690) took 1492 ms. Trace:
I20260709 13:02:12.055029  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.058312 14558 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:12.053272  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.055156  8817 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:12.082517  8824 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:12.084759  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2179) took 1571 ms. Trace:
I20260709 13:02:12.085369  8744 rpcz_store.cc:276] 0709 13:02:10.513628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.122667 (+609039us) service_pool.cc:224] Handling call
0709 13:02:11.122949 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.173777 (+ 50828us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:11.173820 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.197102 (+ 23282us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.598223 (+401121us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:11.598281 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.598374 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.084330 (+485956us) raft_consensus.cc:1685] finished
0709 13:02:12.084357 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.084663 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50420,"raft.queue_time_us":55814,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4417,"thread_start_us":342964,"threads_started":1,"wal-append.queue_time_us":492871}
I20260709 13:02:12.049732 14459 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:12.093896  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.083990  8890 rpcz_store.cc:276] 0709 13:02:10.587846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.201131 (+613285us) service_pool.cc:224] Handling call
0709 13:02:11.240280 (+ 39149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.240499 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:11.610840 (+370341us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.073966 (+463126us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.079017 (+  5051us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.080303 (+  1286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.080352 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:12.080395 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.080462 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.080589 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22198,"prepare.queue_time_us":560402,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":82151,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":796207,"threads_started":4,"wal-append.queue_time_us":319592}
W20260709 13:02:12.097379  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.093930  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.095860  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.098466  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.099855  8817 raft_consensus.cc:2410] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:02:12.081614 14558 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.102634  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.105620  8824 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:12.080303 14579 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:12.109395 14636 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:02:12.109709 14631 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:12.111363  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:12.111927  8893 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.115991  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.123221  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.124375  8824 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:12.114449 14425 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.001s
I20260709 13:02:12.040671 14622 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.132054  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.132768  8818 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:12.133692  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2498) took 1520 ms. Trace:
I20260709 13:02:12.134114  8898 rpcz_store.cc:276] 0709 13:02:10.613529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.690131 (+1076602us) service_pool.cc:224] Handling call
0709 13:02:11.690583 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.691039 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:11.867798 (+176759us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.995578 (+127780us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.018039 (+ 22461us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.018191 (+   152us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:12.018225 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.018299 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.133508 (+115209us) raft_consensus.cc:1685] finished
0709 13:02:12.133528 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.133649 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19091,"prepare.queue_time_us":96911,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":2133,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":13288,"thread_start_us":228546,"threads_started":2,"wal-append.queue_time_us":158698}
I20260709 13:02:12.051708 14650 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.119948 14518 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.136720  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 300505us
I20260709 13:02:12.137389  8893 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:12.140049 14532 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.141438 14425 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.143373  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:12.145706 14651 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:02:12.146387 14628 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:12.146945 14628 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:12.147609 14459 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.148010  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1125) took 3467 ms. Trace:
I20260709 13:02:12.147830 14628 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.148401  9445 rpcz_store.cc:276] 0709 13:02:08.680725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.208323 (+527598us) service_pool.cc:224] Handling call
0709 13:02:11.027103 (+1818780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.042865 (+ 15762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.147818 (+1104953us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54001937d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:12.147950 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4182378752}
W20260709 13:02:12.149776 14640 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.378s	user 0.002s	sys 0.000s
I20260709 13:02:12.151153 14572 raft_consensus.cc:3055] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:12.152849  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:12.156400  8818 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.159674  8891 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:12.160225  8898 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:12.161456  8898 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.162256  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2785) took 1304 ms. Trace:
I20260709 13:02:12.154747 14572 raft_consensus.cc:740] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:12.162671  8823 rpcz_store.cc:276] 0709 13:02:10.858073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.859998 (+  1925us) service_pool.cc:224] Handling call
0709 13:02:12.162027 (+1302029us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54000811d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:12.162168 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239486336}
I20260709 13:02:12.163565 14572 consensus_queue.cc:260] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.164733 14572 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:12.165755  8895 raft_consensus.cc:1240] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:12.141978 14425 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.167444  8896 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:12.168745  8896 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:12.169718  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2419) took 2461 ms. Trace:
I20260709 13:02:12.170097  8740 rpcz_store.cc:276] 0709 13:02:09.708681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.708834 (+   153us) service_pool.cc:224] Handling call
0709 13:02:09.709202 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.171611 (+462409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.169506 (+1997895us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54001c8254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:12.169664 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942800768}
W20260709 13:02:12.174242 14594 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 2.124s	user 0.000s	sys 0.013s
W20260709 13:02:12.175009 14613 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.448s	user 0.006s	sys 0.007s
I20260709 13:02:12.176448  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1056) took 3249 ms. Trace:
I20260709 13:02:12.176859  9443 rpcz_store.cc:276] 0709 13:02:08.926907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.414856 (+487949us) service_pool.cc:224] Handling call
0709 13:02:09.415285 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.574792 (+159507us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:09.674956 (+100164us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.878665 (+203709us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.879227 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.879413 (+   186us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:09.879701 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.886956 (+  7255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.887035 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.381462 (+494427us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4800280158. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:02:10.664662 (+283200us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18002abe00. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:11.202992 (+538330us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b18002abe00. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:12.083271 (+880279us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:12.176213 (+ 92942us) raft_consensus.cc:1685] finished
0709 13:02:12.176237 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.176387 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8643,"prepare.queue_time_us":259346,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":91,"raft.queue_time_us":73241,"raft.run_cpu_time_us":2648,"raft.run_wall_time_us":504964,"spinlock_wait_cycles":30008576,"thread_start_us":379934,"threads_started":3,"wal-append.queue_time_us":213854}
W20260709 13:02:12.185989 14481 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 13:02:12.186448 14481 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.188580 14481 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.144704 14518 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.122510 14580 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.158457 14507 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:12.198671  8712 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.793s (SENT)
I20260709 13:02:12.201099  8820 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.201488  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.201741  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.202668  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.203158  8742 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:12.203233  8823 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:12.203935  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.207325  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.207553  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.207979  8818 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:12.209093 14558 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.208894 14637 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.211349  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.215040  8896 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:12.220841  8817 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:12.222730  8823 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.224292  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.208173 14497 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:02:12.228547  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.260036  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.216789 14637 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.267267 14637 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.154359 14532 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:02:12.261150 13709 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 13:02:12.264700 14652 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:02:12.218823 14610 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:12.283667  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2501) took 1668 ms. Trace:
I20260709 13:02:12.283987 14578 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:12.284435 14578 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:12.285378 14578 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.286693 14578 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.261842 14463 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.289701  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.289579 14463 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.291020  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.291234 13709 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.292009 14463 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.291851 13709 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.292289  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.294003 13709 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.260429  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.271867  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:12.295367  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.296397  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:12.284126  8892 rpcz_store.cc:276] 0709 13:02:10.614750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.717454 (+1102704us) service_pool.cc:224] Handling call
0709 13:02:11.759650 (+ 42196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.768026 (+  8376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.768063 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.783798 (+ 15735us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.877390 (+ 93592us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:11.895439 (+ 18049us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.018618 (+123179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.018695 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.263982 (+245287us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:12.283382 (+ 19400us) raft_consensus.cc:1685] finished
0709 13:02:12.283483 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.283609 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47461,"prepare.queue_time_us":17132,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":93196,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":19219,"spinlock_wait_cycles":12582656,"thread_start_us":203445,"threads_started":2,"wal-append.queue_time_us":260533}
W20260709 13:02:12.296965  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.292984  8823 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:12.298384  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.297444  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.256362  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.258859 14460 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:12.281335  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1141) took 1675 ms. Trace:
I20260709 13:02:12.311106  9447 rpcz_store.cc:276] 0709 13:02:10.606188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.035078 (+428890us) service_pool.cc:224] Handling call
0709 13:02:11.035326 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.321131 (+285805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.321161 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.370303 (+ 49142us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.596040 (+225737us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:11.879856 (+283816us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.895443 (+ 15587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.149232 (+253789us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:12.280942 (+131710us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54006720d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:12.281008 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.281061 (+    53us) raft_consensus.cc:1685] finished
0709 13:02:12.281076 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.281241 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24002,"prepare.queue_time_us":298937,"prepare.run_wall_time_us":4698,"raft.queue_time_us":103336,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":68418,"spinlock_wait_cycles":599192448,"thread_start_us":394765,"threads_started":2,"wal-append.queue_time_us":250336}
W20260709 13:02:12.291369 14634 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.523s	user 0.001s	sys 0.001s
I20260709 13:02:12.264739  8746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "029b57162d72422bba468c7d86b67fca"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:12.314985  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2497) took 1702 ms. Trace:
I20260709 13:02:12.315359  8897 rpcz_store.cc:276] 0709 13:02:10.612968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.169648 (+556680us) service_pool.cc:224] Handling call
0709 13:02:11.170103 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.173892 (+  3789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.173921 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.199585 (+ 25664us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.598606 (+399021us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:11.598668 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.598752 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.290782 (+692030us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:12.314704 (+ 23922us) raft_consensus.cc:1685] finished
0709 13:02:12.314725 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.314926 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":433,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":22059,"spinlock_wait_cycles":4658048,"thread_start_us":329766,"threads_started":1,"wal-append.queue_time_us":359777}
I20260709 13:02:12.295369  8896 raft_consensus.cc:2376] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12. Re-sending same reply.
I20260709 13:02:12.277350  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.302728  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.318378  8824 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.185055 14472 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.318689  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.309999 14408 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:12.310324  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.220057 14558 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.304951  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.320050  8864 leader_election.cc:400] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:02:12.330521  8864 leader_election.cc:403] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:02:12.320711  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 64457.6us
W20260709 13:02:12.331619  8712 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.913s (SENT)
I20260709 13:02:12.332239  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.220460 14573 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.334049  8712 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.931s (SENT)
I20260709 13:02:12.334523  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.335836 13709 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.335948  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:12.332391  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.337348  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:12.334525  8863 leader_election.cc:400] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 11 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
W20260709 13:02:12.351958  8712 leader_election.cc:336] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.977s (SENT)
I20260709 13:02:12.352653  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.355011  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.356899  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.363158  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.374569  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.374993  8744 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:12.376045  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.347715 14573 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.376925 14627 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.377665  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.374929  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.337268 14496 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 11. Current term is 12.
I20260709 13:02:12.339928 14449 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:12.379053  8743 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:12.378827  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.363590 14521 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:12.379932  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.381192  8822 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:12.389798  8748 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:12.390195  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.345674 14432 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.354326 13709 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.340569 14558 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.386787  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:12.426795  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.427268 14558 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.427942  8788 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.428864  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.429621 14610 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.430044 14610 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.430411 14610 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.384757  8744 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:12.434680  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.435199 14616 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:12.435905 14616 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.436085  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:12.436396 14616 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.363116 14408 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:12.389277  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.437254 14408 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.437620  8743 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:12.437708 14408 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:12.437927  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:12.438534  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.438706 14537 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:12.439148  8743 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:12.439611 14537 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:12.440006  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.390674  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.439985 14537 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.441067 14537 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.363399 14411 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:12.456418  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.393427  8746 raft_consensus.cc:606] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.391062 14611 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:12.395046  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:12.462788  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.392889 14432 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.470415 14611 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.470778 14432 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.470966 14611 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.471455 14611 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.471801 14637 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.392812 14472 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.388545 14449 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:12.479501 14611 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.480403 14449 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.363312 14564 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:02:12.397699 14673 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:02:12.398100 14521 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:12.482916  8742 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:12.482810 14521 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.481920 14449 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:12.398559 14573 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.399746  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.402168 14612 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:12.402578  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2709) took 1128 ms. Trace:
I20260709 13:02:12.490651  8789 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.378018 14604 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:02:12.415777 14539 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.422336  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2792) took 1349 ms. Trace:
I20260709 13:02:12.380496 14658 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:02:12.378715 14460 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.425061  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 973) took 1831 ms. Trace:
I20260709 13:02:12.384320 14513 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:12.440156 14616 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:12.443361  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.382213  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.444044  8743 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:12.507752  8892 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:12.507769  8747 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:12.444566  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:12.508924  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:12.509379  8744 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:12.509714  8892 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:12.447446  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.446048  9447 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:12.510571  9443 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:12.453518  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.455163  9442 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.459749  8746 raft_consensus.cc:614] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:12.464310  8896 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:12.465137  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.513249  8748 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:12.380314 14660 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:12.468551  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:12.468974 14540 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.469332  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:12.469563  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.470316  8749 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:12.520949  9449 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.490801  8890 rpcz_store.cc:276] 0709 13:02:11.273778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.137953 (+864175us) service_pool.cc:224] Handling call
0709 13:02:12.157067 (+ 19114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.159076 (+  2009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.159106 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.159570 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.325250 (+165680us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:12.325314 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.325382 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.402345 (+ 76963us) raft_consensus.cc:1685] finished
0709 13:02:12.402364 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.402514 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4952,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":146768,"threads_started":1,"wal-append.queue_time_us":222633}
I20260709 13:02:12.492739  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.496017  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.522701  9446 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:12.523017  9444 raft_consensus.cc:2368] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:12.523949  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.495942  8819 rpcz_store.cc:276] 0709 13:02:11.073106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.073362 (+   256us) service_pool.cc:224] Handling call
0709 13:02:11.114028 (+ 40666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.873134 (+759106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.421995 (+548861us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540012c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:12.422253 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736673664}
I20260709 13:02:12.499163  8816 rpcz_store.cc:276] 0709 13:02:10.593511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.595251 (+  1740us) service_pool.cc:224] Handling call
0709 13:02:10.595563 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.183544 (+1587981us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b1800352780. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64c7bfb 00007f90f64d264b
0709 13:02:12.183704 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.183729 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.184245 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.305955 (+121710us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:12.305996 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.306067 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.424769 (+118702us) raft_consensus.cc:1685] finished
0709 13:02:12.424792 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.425002 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12491,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9071,"spinlock_wait_cycles":4430720,"thread_start_us":145828,"threads_started":2,"wal-append.queue_time_us":208559}
I20260709 13:02:12.503273  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.527122  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.495141 14350 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.491413 14411 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.520251  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:12.480216 14518 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.521173  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:12.457621  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 105719us
I20260709 13:02:12.542213  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.542256  8896 raft_consensus.cc:2376] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17. Re-sending same reply.
I20260709 13:02:12.529811  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.530715  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.499763 14564 consensus_queue.cc:1243] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:12.533775  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.497478 14463 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:02:12.542872  9441 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:12.507366 14616 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:12.546087  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.540693  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:12.548032  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.543407  9445 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:12.555428  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.570232  9449 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:12.551038  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:12.601253  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50305.2us
I20260709 13:02:12.552910  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.571889  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.575187  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.537855 14459 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:12.589608  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43674.6us
I20260709 13:02:12.574103 14513 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:02:12.589687  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.538383 14625 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.591523  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.625174  9441 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:02:12.625346  8712 leader_election.cc:400] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14.
I20260709 13:02:12.543517 14612 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.563977 14627 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.602308  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.625773  8712 leader_election.cc:403] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:12.574779 14353 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:12.626521  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 54777.2us
I20260709 13:02:12.630990  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:12.632655  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:12.629426  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:12.633807  8897 raft_consensus.cc:2376] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12. Re-sending same reply.
I20260709 13:02:12.634958  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.549741 14411 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.544399  8892 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:02:12.643729  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 90920.6us
I20260709 13:02:12.668776  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:12.669471  8890 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:12.590878 14485 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:12.663933  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61735.1us
I20260709 13:02:12.631584 14353 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.574235 14350 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:02:12.636484 14513 consensus_queue.cc:1243] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:12.665123 14663 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.666733  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:12.666750  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.713256  8713 leader_election.cc:400] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:12.714557  8713 leader_election.cc:403] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:02:12.666021  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.629806 14657 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:12.627832 14658 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:12.692021  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.665450 14472 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.667021 14625 raft_consensus.cc:2764] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:12.696880  8897 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.697171  9441 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:12.591747 14564 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.710706  8890 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:12.696702 14539 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14.
W20260709 13:02:12.710135 14683 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:02:12.720817  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.695657 14353 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.603470 14610 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.718256 14658 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.634268 14616 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.723510 14637 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.726063 14684 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:12.718134 14513 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:12.750293 14513 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.718659 14350 consensus_queue.cc:1243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:12.716568 14657 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.753944 14657 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:12.755945 14664 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.696s	user 0.002s	sys 0.001s
I20260709 13:02:12.756525  8745 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:12.746686 14539 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:12.757675 14539 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:12.758615 14539 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.722627  8892 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:12.665856 14518 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:12.762257 14518 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:12.763309 14518 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.764755 14455 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.733606 14663 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.765641 14496 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.765899 14411 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.766178 14496 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.766456 14684 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.767710  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.768330 14496 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:12.739066 14686 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:02:12.740592 14619 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.368s	user 0.006s	sys 0.006s
I20260709 13:02:12.723825 13709 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:12.721740 14472 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.748854  8742 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:02:12.742237 14564 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.743383 14658 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:12.783468  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.730736  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.792047 14572 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:12.749616 14616 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.746176 14625 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:02:12.794340  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.795766  8742 pending_rounds.cc:85] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:12.804829  8742 pending_rounds.cc:107] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:02:12.805318  8742 raft_consensus.cc:2889] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:12.769193 14684 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.802296 14496 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:12.803531  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 72875.2us
I20260709 13:02:12.742654 14537 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:12.809739 14688 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:12.811208 14408 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:12.811710 14662 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:12.797624 14472 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.795150 14628 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:02:12.799597 14564 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:12.807539 14625 consensus_queue.cc:1243] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:12.823465  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.851186  8820 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:12.826576  8824 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:12.826752 14610 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.828591 14650 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:12.830626 14660 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.834445 14513 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.835891  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:12.836724 14425 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:12.837199  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.802944 14353 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.815541 14623 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:12.838973  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2787) took 1924 ms. Trace:
I20260709 13:02:12.896369  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.897060  9440 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:12.896648  8821 rpcz_store.cc:276] 0709 13:02:10.914092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.914237 (+   145us) service_pool.cc:224] Handling call
0709 13:02:10.916855 (+  2618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.917193 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.917221 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.028208 (+110987us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.165225 (+137017us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:11.165272 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.166584 (+  1312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.739901 (+1573317us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:12.838660 (+ 98759us) raft_consensus.cc:1685] finished
0709 13:02:12.838682 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.838893 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154081,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":28126,"spinlock_wait_cycles":70540672,"thread_start_us":136557,"threads_started":1,"wal-append.queue_time_us":239086}
I20260709 13:02:12.898242  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.864634  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:12.865973  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:12.885131  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 358099us
W20260709 13:02:12.888104  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 222166us
I20260709 13:02:12.910110  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:12.912313  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.913259  8817 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:12.850466 14564 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.899187  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:12.924490  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.925151  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:12.929358  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.930732  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.932646  8824 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.933001  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:12.934630  8820 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.842111 14662 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:12.814478 14657 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:12.928900  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 120269us
I20260709 13:02:12.925765 14425 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:12.898599 14637 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:12.918243 14610 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:12.936257  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:12.938566  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:12.898785 14623 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.941087  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:12.971482  8824 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:12.972462 14623 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:12.941597  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:12.893703 14684 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:02:12.967861 13709 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:12.930143 14627 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.926667 14350 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:02:12.932634 14663 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.968410 14540 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:12.968736 14425 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.010349  9441 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:02:12.984504  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48362.9us
I20260709 13:02:12.895428 14660 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.988341  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:12.988440  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:12.988456  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 46958.4us
I20260709 13:02:13.014896  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:12.892400  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 171645us
W20260709 13:02:12.989722  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48729.5us
I20260709 13:02:12.995710 14537 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:12.996555 14408 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
W20260709 13:02:12.998446 14691 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:02:13.006812  8815 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:02:13.022171  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:13.026281  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:13.006219 14350 consensus_queue.cc:1243] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:12.987278 14518 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.017897  9440 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:12.974303 14464 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:02:12.990819 14627 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.975114 14637 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:12.941186 14615 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:13.029398 14602 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:13.030453 14528 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:13.028039 14513 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:13.031481 14684 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:12.976449 14610 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:13.039438  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 173583us
I20260709 13:02:12.993597 14612 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:13.012419 14539 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:13.015372 14425 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:13.041501  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:13.021106 14459 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:13.023483 13709 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:13.019814 14455 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:02:13.062819  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.082940  8892 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:13.069983 14572 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:12.996889 14662 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:13.045660 14627 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.110954 14662 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:13.111831 14685 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.426s	user 0.001s	sys 0.000s
I20260709 13:02:13.071033 14408 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:02:13.072803 14657 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.072748 14602 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:13.071673 14539 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.074051 14353 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.076982  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.052323 14580 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:13.122604  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:13.077555  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50251.5us
W20260709 13:02:13.077298  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 90196.9us
I20260709 13:02:13.078287  8894 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:13.040992 14694 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:13.083753  8821 raft_consensus.cc:2410] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:02:13.038759 14485 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:02:13.085913 14693 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:02:13.087177  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2199) took 1086 ms. Trace:
W20260709 13:02:13.090507 14432 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.002s
I20260709 13:02:13.091990 14564 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.194520  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 153107us
I20260709 13:02:13.028873 14472 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.093005 14507 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.093892 13709 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.094491  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 997) took 1189 ms. Trace:
I20260709 13:02:13.095827 14610 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.097600 14660 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.060187 14703 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 13:02:13.098781 14449 consensus_meta.cc:287] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.002s
W20260709 13:02:13.101907 14699 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:13.067252 14612 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.109246  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.058041 14658 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.118199  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 93821.8us
I20260709 13:02:13.195454  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:13.195497  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62586.7us
I20260709 13:02:13.172866  8822 rpcz_store.cc:276] 0709 13:02:12.000614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.001159 (+   545us) service_pool.cc:224] Handling call
0709 13:02:12.001498 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.391991 (+390493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.086889 (+694898us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540040a454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:13.087115 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872748800}
I20260709 13:02:13.379570 14612 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:13.260695  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 184368us
I20260709 13:02:13.331326 14463 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:13.381115  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:13.381575 14716 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:13.281255 14602 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.386464 14716 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:13.386058 14463 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:13.386333  9448 rpcz_store.cc:276] 0709 13:02:11.905274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.919369 (+ 14095us) service_pool.cc:224] Handling call
0709 13:02:11.919573 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.063572 (+143999us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:12.131105 (+ 67533us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.553326 (+422221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.553381 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:12.553421 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.094255 (+540834us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54005b7554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:13.094303 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.094435 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10927,"prepare.queue_time_us":71997,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":19011,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"spinlock_wait_cycles":330443008,"thread_start_us":450262,"threads_started":2,"wal-append.queue_time_us":469108}
I20260709 13:02:13.387703 14716 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.387897 14463 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.121783 14580 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.386101  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.389017 14580 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.390947 14658 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.391458 14557 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:13.392004 14557 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:13.392481  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.315953 14637 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.393613 14713 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.389012 14463 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.197614 14350 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.197s
I20260709 13:02:13.395531 14610 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:13.307282 14711 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:02:13.396179  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:13.394266 14557 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.398387  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:13.382077  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 186720us
I20260709 13:02:13.399631  8894 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:13.400036 14507 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:13.400417 14572 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:13.400614 14507 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.400976 14572 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.403151  8894 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:13.403260 14572 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.404088 14507 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.399322  8818 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:13.406621  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.406467 14644 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:13.408252  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.411235  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.409046  8818 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:13.414844 14657 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.415069  8898 raft_consensus.cc:1217] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[14.9-18.11]   Dedup: 14.9->[16.10-18.11]
I20260709 13:02:13.415589 14657 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.198721 14432 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:13.418704  8896 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:13.419807 14657 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.415601  8898 pending_rounds.cc:85] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:02:13.420728 14432 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.421228  8898 pending_rounds.cc:107] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:02:13.421666  8898 raft_consensus.cc:2889] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:13.422417 14557 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:13.122409 14539 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:13.125101 14694 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:13.425858 14694 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.292373 14623 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.431458 14572 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:13.424777 14612 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:13.349192 14712 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.434805 14657 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:13.434742 14712 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:13.435249  8824 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:13.394268 14637 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:13.436777 14612 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.436821 14637 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.440936  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.470209  8817 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:13.473688  9443 raft_consensus.cc:1217] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Deduplicated request from leader. Original: 9.8->[11.9-13.10]   Dedup: 13.10->[]
I20260709 13:02:13.474383  9443 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 13:02:13.481205  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:13.492096  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.500403  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:13.471807  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:13.447104  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:13.447554  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.459254  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:13.303318 14698 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 13:02:13.303990 14654 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.253s	user 0.003s	sys 0.003s
I20260709 13:02:13.508796  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2450) took 1050 ms. Trace:
I20260709 13:02:13.509159  8745 rpcz_store.cc:276] 0709 13:02:12.458482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.757350 (+298868us) service_pool.cc:224] Handling call
0709 13:02:12.757672 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.758007 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.758036 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.758515 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.967394 (+208879us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:12.967446 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.969638 (+  2192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.508616 (+538978us) raft_consensus.cc:1685] finished
0709 13:02:13.508635 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.508744 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1224,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":50,"thread_start_us":208424,"threads_started":1,"wal-append.queue_time_us":264679}
I20260709 13:02:13.462291  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.502377  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:13.511235  8822 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:13.511823  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:13.512095 14612 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:13.512576  8898 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:13.509025  8816 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:13.514578  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67576.9us
I20260709 13:02:13.515012  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:13.515671  9448 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:13.504546  9446 pending_rounds.cc:85] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 12
I20260709 13:02:13.517800  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:13.518265  9446 pending_rounds.cc:107] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:02:13.518549  9446 raft_consensus.cc:2889] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:13.467998 14703 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.521054  9445 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:13.526747  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:13.501346 14623 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.533493  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:13.514791 14564 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.541816  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:13.543185  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 71508.1us
I20260709 13:02:13.500129 14637 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.535270 14719 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:02:13.556697  9445 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20.
I20260709 13:02:13.521044 14472 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.532531 14709 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:13.565659  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:13.566682  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 95494.3us
I20260709 13:02:13.569239  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.570155  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:13.570745  9441 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.577028  8788 leader_election.cc:400] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20.
I20260709 13:02:13.577543  8788 leader_election.cc:403] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:13.556529 14696 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:13.575464  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.581005  9445 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:13.584853  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:13.583091  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:13.583997 14696 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.563922 14709 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:13.532800 14703 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.574331 14604 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.590920 14472 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.591159 14662 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:13.602152 14696 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.630211 14733 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:13.604048 14709 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:13.626520  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59544.7us
I20260709 13:02:13.604612 14485 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.643445 14735 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:13.644284 14735 consensus_queue.cc:1243] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:02:13.644477 14425 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.009s
I20260709 13:02:13.644928 14425 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.645511 14726 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:02:13.647158 14731 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:13.647398 14425 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:13.650063 14732 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:02:13.652056 14722 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:13.653558  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2214) took 1558 ms. Trace:
I20260709 13:02:13.653964  8815 rpcz_store.cc:276] 0709 13:02:12.095464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.095610 (+   146us) service_pool.cc:224] Handling call
0709 13:02:13.653335 (+1557725us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54002120d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:13.653488 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086536960}
I20260709 13:02:13.659476  9448 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:13.659667 14703 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.661804  9441 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
W20260709 13:02:13.663965  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 1.33351e+06us
I20260709 13:02:13.664745  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.645265  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:13.673139  8788 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:13.675971  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:13.678614  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56580 (request call id 2) took 1423 ms. Trace:
I20260709 13:02:13.679155  8746 rpcz_store.cc:276] 0709 13:02:12.223917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.224158 (+   241us) service_pool.cc:224] Handling call
0709 13:02:13.641366 (+1417208us) spinlock_profiling.cc:243] Waited 78.8 ms on lock 0x7b54001d9f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64c9853 00007f90fb49760a 00007f90f4a1d7d4 00007f90f4a1d762 00007f90f4a1d691 00007f90f4a1d60c 00007f90f4a1c8c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:13.641524 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181286656}
I20260709 13:02:13.681965  8892 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:13.684907  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:13.689496  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:13.663528 14520 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:13.621522 14604 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.712788  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 137446us
I20260709 13:02:13.714824  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:13.675196 14606 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:13.716686  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:13.686425 14615 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:13.731303  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.742699  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.744118  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:13.709488 14520 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.753201  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 167096us
I20260709 13:02:13.727907 14573 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:13.727062 14606 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:13.728931 14518 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.707893 14703 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:02:13.729840 14739 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.737947 14425 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.755949 14520 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.750473 14615 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:13.727335 14580 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.760740 14573 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:13.799969 14539 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.806430 14741 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:13.811388 14518 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.813890 14615 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.815469 14606 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.786760 14716 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.819126  8740 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:13.818682 14694 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.821260 14728 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:02:13.823954 14643 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:13.839977  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 150556us
I20260709 13:02:13.865957  8740 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:13.860525 14741 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.871606  8740 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:13.877585  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 290751us
W20260709 13:02:13.879710  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 120804us
I20260709 13:02:13.882615  8740 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.880055  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:13.890121  8789 connection.cc:256] RPC call timeout handler was delayed by -1.2503193109999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 13:02:13.891815  8740 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:13.888267 14612 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.895538  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:13.895924  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:13.896767  8742 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.863454 14742 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:13.902426  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:13.906890  8743 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.895176 14612 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.908947 14612 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.903275  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:13.909502  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:13.883754 14627 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.001s
I20260709 13:02:13.910351 14627 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:13.911482 14703 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.911677  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:13.912022 14703 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.912447 14627 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.909770  8748 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:13.909380 14540 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:13.913309  8746 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:13.914186 14703 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.916608  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:13.917353  9447 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:13.920347 14713 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:13.920912  8743 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:13.923554 14353 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:13.927678  8819 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:13.932716  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.932996  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.933393  8815 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:13.933585 14571 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.934247  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:13.934983  8822 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:13.938886 14539 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.941977  8815 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:13.945538 14573 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.947019 14573 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.949002  8815 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:13.886456 14709 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.952800  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:13.879165 14741 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.954541 14604 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:13.871418 14716 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:13.958137 14580 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:13.960580  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:13.961668  8743 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:13.973392  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 93415.5us
I20260709 13:02:13.974529  8713 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.976243  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:13.981083  8822 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:13.976567  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:13.903218 14558 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 13:02:14.001040  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.953019 14709 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:13.988744  8748 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:14.004832  9447 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:14.011157  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.036561  8822 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:14.067709  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:14.078610  8741 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:13.998041 14580 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.002413 14662 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20.
I20260709 13:02:14.068121 14753 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.084684 14463 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.005975 14759 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:13.986639 14742 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:14.093351  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:14.130837  8822 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:13.975634 14518 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:14.138641  8748 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:14.138967  8822 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:14.144912  8819 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:14.140609  8742 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:14.003957 14481 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:14.142956  8816 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:02:14.089900 14716 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.150166  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:14.150166  8816 pending_rounds.cc:85] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 12
I20260709 13:02:14.153756  8816 pending_rounds.cc:107] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:02:14.154045  8816 raft_consensus.cc:2889] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:14.151513  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:14.079888 14580 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:14.163826  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.959s (SENT) [suppressed 33 similar messages]
I20260709 13:02:14.141049 14643 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.164276  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:14.169605  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.169734  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.170451  8748 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:14.170653  8742 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:14.172803  8742 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:14.129935 14463 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.173925  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:14.178815  8818 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:14.170082  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:14.177837  8746 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:14.191176  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.196636  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:14.193230  8746 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:14.175372 14758 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:02:14.199807  8819 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:14.199744 14703 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:02:14.171653 14759 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:14.257042  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:14.257967  8746 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:14.262351  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53650.2us
I20260709 13:02:14.263733  8815 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:14.264257  8815 pending_rounds.cc:85] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 11
I20260709 13:02:14.264604  8815 pending_rounds.cc:107] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:14.265101  8815 raft_consensus.cc:2889] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:14.165459 14481 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.270803  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:14.171176 14643 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.272229  8891 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:14.200104 14716 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:02:14.205428 14579 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:02:14.202617 14694 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.285566 14532 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:14.289016  8747 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:14.289624 14573 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:14.221688 14628 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 13:02:14.291373 14696 consensus_meta.cc:287] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 13:02:14.273455 14622 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 13:02:14.305778  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 131936us
I20260709 13:02:14.307214  8819 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:14.309510  8891 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:14.206005 14578 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:02:14.328269  8746 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
W20260709 13:02:14.323738  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 159927us
I20260709 13:02:14.205402 14742 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.289124 14612 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.313731 14520 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.001s
I20260709 13:02:14.385097  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.386700  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:14.396636  8891 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:14.400391  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:14.362429 14623 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:14.387018  8746 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:14.407919 14350 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:14.402009 14579 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:14.402592 14623 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:14.415798 14579 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:14.416042 14472 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.788s	user 0.010s	sys 0.002s
I20260709 13:02:14.427531  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:14.425537  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.274852 14759 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.402647 14716 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:02:14.403774 14703 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:02:14.384732 14696 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.416676 14350 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:14.431223  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.417449 14573 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:14.455991  8746 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 17 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:14.426841 14628 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.402364 14612 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.430758 14579 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:14.462527  8746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:14.464665  8746 raft_consensus.cc:606] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:02:14.452236 14472 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:14.415198 14520 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.505631 14606 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:14.506971 14520 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.507617 14520 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:14.427958 14602 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:02:14.509003  8744 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:14.509068  9446 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:14.466943 14350 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.511453 14350 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:14.473814 14573 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.476734 14612 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.459683 14760 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.484226  9440 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:14.463778 14771 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.487531 14695 log.cc:927] Time spent T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.502s	user 0.003s	sys 0.000s
I20260709 13:02:14.490267 14697 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:14.494093 14637 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.501176  8746 raft_consensus.cc:614] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:14.455864 14425 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.454357 14733 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:14.461259 14765 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:02:14.507900 14759 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.471542 14712 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.513353  8823 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:14.524252 14612 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.478905  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:14.516490  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:14.461488 14463 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.485181 14481 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.464079 14580 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.464427 14703 consensus_queue.cc:1243] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:14.548170  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2451) took 2089 ms. Trace:
I20260709 13:02:14.512352 14694 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:14.590174  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73821.3us
I20260709 13:02:14.599679  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:14.591393  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 112575us
I20260709 13:02:14.524400 14643 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:14.570088 14697 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:14.625943  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:14.596210  8740 rpcz_store.cc:276] 0709 13:02:12.458641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.757793 (+299152us) service_pool.cc:224] Handling call
0709 13:02:12.758052 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.892284 (+1134232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.547836 (+655552us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000b5c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:14.548090 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440145792}
I20260709 13:02:14.620244  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:14.620926  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 154815us
I20260709 13:02:14.528545 14696 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.691517  8742 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:14.693135  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:14.696643  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.697431  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.698259  8746 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:14.698395  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.602108  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:14.700119  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 98102.8us
W20260709 13:02:14.702595  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76773.2us
I20260709 13:02:14.596390 14637 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.700700  8740 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:14.707161 14650 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:14.708184 14611 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.595270 14716 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:14.716037 14716 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:14.694286 14580 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:14.717644 14580 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.718019 14580 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:14.538261 14579 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.721396  9447 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:14.722103 14481 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:14.722558 14579 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:14.723465 14741 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.005s
I20260709 13:02:14.723403 14697 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.724265 14741 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.578756 14425 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.710731 14463 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:14.586109 14612 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.579393 14733 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.534871 14771 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.714736  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:14.731053  8818 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:14.694401 14694 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.702687  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:14.555025 14760 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.710018 14643 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.713421 14650 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:14.733347 14697 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.734475 14463 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:14.743650 14733 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.735728 14572 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.717044 14716 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.743690 14771 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.743705 14425 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:14.750816 14785 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
I20260709 13:02:14.752794 14760 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.747605 14572 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.754899 14572 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.745921 14650 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.757567 14612 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.759635 14694 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.761425 14741 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:14.806042  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 320278us
W20260709 13:02:14.817847  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.824225  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:14.828171  8740 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:14.826467  8741 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:14.829949  8745 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:14.830469  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.831082  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.834335  8741 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:14.835466  8741 raft_consensus.cc:606] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:14.836513  8748 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
W20260709 13:02:14.837395  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 134657us
W20260709 13:02:14.838543  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.838743  8824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2239) took 1776 ms (client timeout 1505 ms). Trace:
W20260709 13:02:14.839421  8824 rpcz_store.cc:269] 0709 13:02:13.062431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.062567 (+   136us) service_pool.cc:224] Handling call
0709 13:02:14.838156 (+1775589us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540028b754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:14.838433 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808860544}
I20260709 13:02:14.836064  8741 raft_consensus.cc:614] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:14.842562  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2805) took 1933 ms. Trace:
I20260709 13:02:14.842064  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2529) took 1331 ms. Trace:
I20260709 13:02:14.843284  8817 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:14.843446  8889 rpcz_store.cc:276] 0709 13:02:13.509997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.511288 (+  1291us) service_pool.cc:224] Handling call
0709 13:02:13.511566 (+   278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:13.513956 (+  2390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.513984 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:14.472986 (+959002us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:02:14.473628 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 13:02:14.733546 (+259918us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:14.733588 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.733660 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.837332 (+103672us) raft_consensus.cc:1685] finished
0709 13:02:14.837386 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.841663 (+  4277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2052,"raft.queue_time_us":2424,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5285,"spinlock_wait_cycles":50183424,"thread_start_us":259455,"threads_started":1,"wal-append.queue_time_us":283260}
I20260709 13:02:14.838769  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:14.843127  8821 rpcz_store.cc:276] 0709 13:02:12.908707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.909118 (+   411us) service_pool.cc:224] Handling call
0709 13:02:14.842025 (+1932907us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540024a9d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:14.842363 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353178496}
I20260709 13:02:14.862998  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2518) took 1467 ms. Trace:
I20260709 13:02:14.863211  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.864537  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.870081 14463 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.876287  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2804) took 1978 ms. Trace:
I20260709 13:02:14.877732 14612 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:14.878338 14612 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.878741 14612 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.879664 14703 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:14.880606 14703 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.876767  8823 rpcz_store.cc:276] 0709 13:02:12.897860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.898002 (+   142us) service_pool.cc:224] Handling call
0709 13:02:14.875941 (+1977939us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540012b954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:14.876128 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3712926464}
I20260709 13:02:14.887115  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.888386  9446 raft_consensus.cc:2410] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:02:14.890218  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.891201 14694 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:14.891932 14694 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:14.863693  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:14.863644  8896 rpcz_store.cc:276] 0709 13:02:13.395268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.395887 (+   619us) service_pool.cc:224] Handling call
0709 13:02:14.862371 (+1466484us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b540032b254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:14.862664 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49279872}
I20260709 13:02:14.894559  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:14.895125 14353 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:14.901656  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.903244 14771 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:14.904241 14771 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.895869 14353 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:14.896341 14694 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:14.911305 14623 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.916663  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:14.869160  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:14.917424 14353 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.923321  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.926765 14528 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.926951  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:14.927651 14528 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:14.928354  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:14.914354 14771 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.934816  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:14.920450 14623 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:14.939414 14637 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:14.940191 14637 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:14.944383 14777 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.330s	user 0.000s	sys 0.003s
I20260709 13:02:14.944718 14637 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:14.946910 14528 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:14.948966 14459 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:14.966929 14459 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:14.968281 14459 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:14.978041  8645 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 271) took 47115 ms (client timeout 15000 ms). Trace:
W20260709 13:02:14.978313  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 109520us
I20260709 13:02:14.963307 14532 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.990417  8740 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:14.953832 14697 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:02:14.976917 14459 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:15.012042  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46664.4us
I20260709 13:02:15.039484 14571 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:15.020084 14792 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:02:15.028578  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56841.1us
I20260709 13:02:15.042824 14604 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.054644  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46763.5us
I20260709 13:02:15.053182  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2528) took 1533 ms. Trace:
I20260709 13:02:15.081518 14604 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.083956 14572 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:15.087649 14768 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.088642 14768 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.098672 14459 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.104162 14716 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.105020 14716 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.107156  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.107848 14768 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.111150  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2525) took 1670 ms. Trace:
I20260709 13:02:15.114001 14643 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.114583 14622 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:15.115017 14622 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:15.114751 14643 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.116353 14622 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.100984 14459 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.145188  8740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
W20260709 13:02:15.124971  8646 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 198) took 70468 ms (client timeout 15000 ms). Trace:
I20260709 13:02:15.111614  8895 rpcz_store.cc:276] 0709 13:02:13.440167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.440284 (+   117us) service_pool.cc:224] Handling call
0709 13:02:13.440639 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:13.441039 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.441070 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:13.441816 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 13:02:13.704943 (+263127us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:13.725716 (+ 20773us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.967595 (+1241879us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b180049afe0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:02:15.003776 (+ 36181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.012517 (+  8741us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.027440 (+ 14923us) raft_consensus.cc:1685] finished
0709 13:02:15.073363 (+ 45923us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.110964 (+ 37601us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11211,"prepare.queue_time_us":22819,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1341639,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":32819,"spinlock_wait_cycles":7238400,"thread_start_us":219639,"threads_started":2,"wal-append.queue_time_us":250831,"wal-append.run_cpu_time_us":2654,"wal-append.run_wall_time_us":1116299}
I20260709 13:02:15.127686 14622 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:15.072108 14703 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.157254  8818 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:15.160571  8740 raft_consensus.cc:606] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:15.070752  8890 rpcz_store.cc:276] 0709 13:02:13.509699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.510706 (+  1007us) service_pool.cc:224] Handling call
0709 13:02:13.511030 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.520588 (+  9558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.520625 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.844134 (+1323509us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:02:14.844927 (+   793us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.897598 (+ 52671us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:14.905733 (+  8135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.920865 (+ 15132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.042851 (+121986us) raft_consensus.cc:1685] finished
0709 13:02:15.042876 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.043190 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4821,"raft.queue_time_us":2567,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":10652,"spinlock_wait_cycles":35552000,"thread_start_us":52182,"threads_started":1,"wal-append.queue_time_us":108914}
I20260709 13:02:15.117261 14643 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.168807  8740 raft_consensus.cc:614] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:15.161686  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 1.19123e+06us
I20260709 13:02:15.158684 14572 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:15.174969  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.183233  8823 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.185492 14496 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.186192 14496 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.187873  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1009) took 1627 ms. Trace:
I20260709 13:02:15.188966 14496 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.189613 14481 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.188262  8898 rpcz_store.cc:276] 0709 13:02:13.560407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.909423 (+349016us) service_pool.cc:224] Handling call
0709 13:02:13.909697 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.909885 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:13.910168 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.177560 (+1267392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.183478 (+  5918us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:15.187433 (+  3955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.187550 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.187695 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1025187,"prepare.queue_time_us":769,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":1117033,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":217376,"threads_started":2,"wal-append.queue_time_us":92850,"wal-append.run_cpu_time_us":1491,"wal-append.run_wall_time_us":1001482}
W20260709 13:02:15.190816  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 74368.3us
I20260709 13:02:15.191435 14604 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.196569 14612 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.201370  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:15.204268  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.204067 14579 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:15.204814 14703 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:15.205646  8896 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:15.206485  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.207458  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:15.148324  8646 rpcz_store.cc:269] 0709 13:01:04.656264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.676128 (+ 19864us) service_pool.cc:224] Handling call
0709 13:02:15.124753 (+70448625us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:08.275349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:08.550672 (+275323us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:08.586831 (+ 36159us) write_op.cc:435] Acquired schema lock
0709 13:02:08.603880 (+ 17049us) tablet.cc:662] Decoding operations
0709 13:02:08.677994 (+ 74114us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:08.678085 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:08.678116 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:02:08.678526 (+   410us) tablet.cc:701] Row locks acquired
0709 13:02:08.678572 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:02:08.678729 (+   157us) write_op.cc:270] Start()
0709 13:02:08.678901 (+   172us) write_op.cc:276] Timestamp: P: 1783602128678670 usec, L: 0
0709 13:02:08.678943 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:02:08.679601 (+   658us) log.cc:844] Serialized 919 byte log entry
0709 13:02:09.173442 (+493841us) op_driver.cc:464] REPLICATION: finished
0709 13:02:09.530615 (+357173us) write_op.cc:301] APPLY: starting
0709 13:02:09.543916 (+ 13301us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:09.654249 (+110333us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:09.654376 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:09.655849 (+  1473us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:09.656131 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:02:09.656204 (+    73us) write_op.cc:312] APPLY: finished
0709 13:02:09.656515 (+   311us) log.cc:844] Serialized 33 byte log entry
0709 13:02:09.661488 (+  4973us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.661586 (+    98us) write_op.cc:454] Released schema lock
0709 13:02:09.661650 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:09.663602 (+  1952us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:09.663637 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6789734,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":315873,"apply.run_cpu_time_us":6138,"apply.run_wall_time_us":145335,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12483,"num_ops":2,"prepare.queue_time_us":507448,"prepare.run_cpu_time_us":28776,"prepare.run_wall_time_us":594296,"raft.queue_time_us":132555,"raft.run_cpu_time_us":21760,"raft.run_wall_time_us":389062,"replication_time_us":423441,"thread_start_us":632712,"threads_started":4,"wal-append.queue_time_us":216881,"wal-append.run_cpu_time_us":5954,"wal-append.run_wall_time_us":1754853}]]}
I20260709 13:02:15.208901 14572 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.210624  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2527) took 1763 ms. Trace:
I20260709 13:02:15.210915 14800 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.211323 14411 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.210624 14799 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:14.978778  8645 rpcz_store.cc:269] 0709 13:01:27.862740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.884956 (+ 22216us) service_pool.cc:224] Handling call
0709 13:02:14.977697 (+47092741us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:01.021929 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:01.022989 (+  1060us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:01.023068 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:02:01.023098 (+    30us) tablet.cc:662] Decoding operations
0709 13:02:01.110942 (+ 87844us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:01.132537 (+ 21595us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:01.135186 (+  2649us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:02:01.205065 (+ 69879us) tablet.cc:701] Row locks acquired
0709 13:02:01.205111 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:02:01.205277 (+   166us) write_op.cc:270] Start()
0709 13:02:01.205443 (+   166us) write_op.cc:276] Timestamp: P: 1783602121205216 usec, L: 0
0709 13:02:01.205486 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:02:01.220658 (+ 15172us) log.cc:844] Serialized 7486 byte log entry
0709 13:02:01.868230 (+647572us) op_driver.cc:464] REPLICATION: finished
0709 13:02:02.312247 (+444017us) write_op.cc:301] APPLY: starting
0709 13:02:02.329215 (+ 16968us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:02.954613 (+625398us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:02.954781 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:03.764370 (+809589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:03.819846 (+ 55476us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:02:03.819929 (+    83us) write_op.cc:312] APPLY: finished
0709 13:02:03.820401 (+   472us) log.cc:844] Serialized 172 byte log entry
0709 13:02:04.056890 (+236489us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:04.106952 (+ 50062us) write_op.cc:454] Released schema lock
0709 13:02:04.122853 (+ 15901us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:04.148943 (+ 26090us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:04.148981 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7983457,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":403819,"apply.run_cpu_time_us":91466,"apply.run_wall_time_us":1853805,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83509,"num_ops":19,"prepare.queue_time_us":132016,"prepare.run_cpu_time_us":45873,"prepare.run_wall_time_us":294450,"raft.queue_time_us":310252,"raft.run_cpu_time_us":50436,"raft.run_wall_time_us":359451,"replication_time_us":648731,"thread_start_us":830940,"threads_started":5,"wal-append.queue_time_us":411594,"wal-append.run_cpu_time_us":29627,"wal-append.run_wall_time_us":2384555}]]}
I20260709 13:02:15.186600  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:15.117683 14606 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:02:15.234186  8891 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:15.205686 14716 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.168480 14528 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:02:15.222374  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:15.252705  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.218674 14481 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.268744  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:15.270118  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 88739.6us
I20260709 13:02:15.271358  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2526) took 1806 ms. Trace:
W20260709 13:02:15.269996  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48753.3us
I20260709 13:02:15.271051 14611 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:15.224457  8897 rpcz_store.cc:276] 0709 13:02:13.447256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.447399 (+   143us) service_pool.cc:224] Handling call
0709 13:02:13.447720 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.447967 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:13.448267 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.284201 (+835934us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b180005aee0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:02:15.168411 (+884210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.178924 (+ 10513us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:15.210156 (+ 31232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.210264 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.210446 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":612571,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1906,"raft.queue_time_us":740044,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":43031808,"thread_start_us":219657,"threads_started":2,"wal-append.queue_time_us":940803}
I20260709 13:02:15.277966  8889 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:15.226723 14643 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:02:15.277966  8892 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:15.271896  8893 rpcz_store.cc:276] 0709 13:02:13.446416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.446592 (+   176us) service_pool.cc:224] Handling call
0709 13:02:13.505151 (+ 58559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.505395 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:13.755322 (+249927us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.173818 (+1418496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.181009 (+  7191us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:15.219681 (+ 38672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.221979 (+  2298us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.252453 (+ 30474us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1101014,"prepare.queue_time_us":24235,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3318,"raft.queue_time_us":1241976,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":152,"thread_start_us":534130,"threads_started":3,"wal-append.queue_time_us":124440,"wal-append.run_cpu_time_us":1222,"wal-append.run_wall_time_us":1001211}
I20260709 13:02:15.284559 14796 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:15.204157  8641 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 13, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 13 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:15.285336 14796 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.286799 14703 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:15.287438 14703 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.286768  8641 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.287900 14703 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.288268  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.271085 14799 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:15.287472  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101215us
I20260709 13:02:15.271086  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.289260  8895 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:15.317425 14804 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.317698 14579 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:02:15.322036  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.326457 14803 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:15.332176  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61244.6us
W20260709 13:02:15.306629 14580 consensus_meta.cc:287] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.006s
I20260709 13:02:15.330942  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.288723  8641 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:15.317413 14753 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:15.279876 14800 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:15.343991  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 73625.5us
W20260709 13:02:15.350620  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51400.4us
I20260709 13:02:15.306056 14604 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:15.345973  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:15.348991 14528 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:15.361503 14528 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.361903 14528 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.364323  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:15.372035 14572 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.345533 14580 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.387199  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.399366 14812 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.400272 14812 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.386695 14804 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.410151 14804 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.412842  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.413391 14578 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:15.413693 14604 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.416007 14612 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.416237 14604 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.417145 14612 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.417960 14612 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.418836 14637 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.419416 14637 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:15.419888  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78487.3us
I20260709 13:02:15.422413  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:15.424688 14796 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:15.425374 14816 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.425436  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2465) took 1004 ms. Trace:
I20260709 13:02:15.426357 14816 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.426532  8744 rpcz_store.cc:276] 0709 13:02:14.421059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.510524 (+ 89465us) service_pool.cc:224] Handling call
0709 13:02:14.510833 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.829856 (+319023us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:14.830646 (+   790us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.131747 (+301101us) spinlock_profiling.cc:243] Waited 881 us on lock 0x7b180043c300. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:02:15.132334 (+   587us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.140560 (+  8226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.140624 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:15.145081 (+  4457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.207141 (+ 62060us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b180043c300. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:15.425028 (+217887us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400466ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:15.425075 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.425204 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27586,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":92,"raft.queue_time_us":17891,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":20110,"spinlock_wait_cycles":767111936,"thread_start_us":29882,"threads_started":1,"wal-append.queue_time_us":100520}
I20260709 13:02:15.428681 14816 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.429301  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:15.430390 14643 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:15.430918 14643 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:15.431880 14643 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.432744  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:15.433535  8824 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:15.434166  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1010) took 1867 ms. Trace:
I20260709 13:02:15.434566  8894 rpcz_store.cc:276] 0709 13:02:13.566677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.036784 (+470107us) service_pool.cc:224] Handling call
0709 13:02:14.037171 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:15.003602 (+966431us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:15.159042 (+155440us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.268757 (+109715us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:15.341559 (+ 72802us) log.cc:844] Serialized 52 byte log entry
0709 13:02:15.341814 (+   255us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:15.341865 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.349740 (+  7875us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.433632 (+ 83892us) raft_consensus.cc:1685] finished
0709 13:02:15.433662 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.433914 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":952751,"prepare.queue_time_us":230658,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4533,"raft.queue_time_us":1103671,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":2874,"thread_start_us":237582,"threads_started":3,"wal-append.queue_time_us":128362}
W20260709 13:02:15.439020  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 74794.6us
W20260709 13:02:15.439313 14810 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:02:15.441625  8641 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:15.443128 14798 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:02:15.444002  8641 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 15, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 15 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.464432 14481 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.451181 14800 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:15.373534 14808 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:02:15.470090 14650 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.472996 14812 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.455109 14580 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.455915 14803 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:15.457118  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2466) took 1035 ms. Trace:
I20260709 13:02:15.474653  8746 rpcz_store.cc:276] 0709 13:02:14.421746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.699034 (+277288us) service_pool.cc:224] Handling call
0709 13:02:14.699397 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.704197 (+  4800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.704242 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.922565 (+218323us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.065564 (+142999us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:15.065621 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.065742 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.442273 (+376531us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:15.456482 (+ 14209us) raft_consensus.cc:1685] finished
0709 13:02:15.456518 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.456886 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242210,"raft.queue_time_us":6152,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":238156,"spinlock_wait_cycles":31936640,"thread_start_us":101723,"threads_started":1,"wal-append.queue_time_us":210399}
I20260709 13:02:15.370625 14611 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:15.477301 14611 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:15.477458 14812 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.478225 14812 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.478911 14611 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.480407 14812 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.480786 14611 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.449196 14411 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:15.460296 14528 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.461200  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.461794 14637 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:15.462668 14623 consensus_meta.cc:287] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.005s
I20260709 13:02:15.464063  8641 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:15.445914 14716 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:15.466135 14799 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:15.458168  8819 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:15.484314 14738 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:15.485127 14571 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:15.485632 14703 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.486459  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64174.2us
I20260709 13:02:15.488023 14612 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.413113 14804 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.488709 14622 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Committing config change with OpId 15.10: config changed from index 2 to 10, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.501039 14796 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.500994  8889 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:15.503206 14411 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:15.528342  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.529197 14528 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.530071 14805 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.530653 14805 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.531198  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 206172us
I20260709 13:02:15.532413 14796 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.533403  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:15.533965 14805 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.534889 14796 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.503877 14717 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:02:15.503990  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 119188us
I20260709 13:02:15.541144  8744 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:15.541465 14753 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:15.506534 14520 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:15.542747 14520 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:15.543296  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.507261 14716 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:15.544694 14712 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:15.545137  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:15.545140 14712 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:15.545814  8816 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.546378 14712 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.546912 14703 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:15.504112 14814 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.112s	user 0.002s	sys 0.004s
I20260709 13:02:15.543725 14520 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.547649 14712 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.550078  8746 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:15.545174 14716 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.507889  8641 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:15.555634  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.556423  8742 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:15.556284  8641 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 13, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 13 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:15.558081  8641 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 12, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 12 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:15.560719  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.561652  8740 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:15.566697 14717 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:15.570914  8816 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:15.570201 14804 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.576785  8824 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:15.576968  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 239380us
I20260709 13:02:15.566733 14796 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:15.584118  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:15.588732  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 236019us
I20260709 13:02:15.585057  8740 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
W20260709 13:02:15.599261  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66055.8us
I20260709 13:02:15.611881  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.596330 14353 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:15.615063  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 253050us
I20260709 13:02:15.566112  8641 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 11, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 11 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.605610 14716 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.593685 14804 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.626286 14612 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.629665  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:15.654546  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:15.662447  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.663491  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.666194  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:15.618610 14637 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:15.675802  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.677204  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.677379  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.656239 14528 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:15.678069  8816 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 12.
I20260709 13:02:15.680351  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.619761 14353 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:15.681145  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.656765 14812 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.682821  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.680968  8743 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:15.683038  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:15.682333  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:15.690415  8745 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.688956  8748 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:15.692214 14472 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:02:15.693852 14828 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:15.689747  8741 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:15.697939  8824 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.699057 14829 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.699141 14828 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:15.699653 14829 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.700325 14828 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.703101  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:15.703701  8747 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:15.706183  8823 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:15.706773 14353 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:15.708190 14820 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:15.708866 14741 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.709465 14637 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.709527 14741 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.711870 14637 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.610456 14717 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.684113 14528 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.697037 14697 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.724992 14697 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.726552  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:15.728282 14703 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:15.728732 14703 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.729175 14703 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.696844 14812 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.731156 14812 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.694034 14615 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.731863 14615 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.715114  8815 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:15.733613 14804 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.734059 14615 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:15.734567  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.717507 14738 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:15.732098  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:15.735570  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 118506us
I20260709 13:02:15.718956 14472 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:15.713075  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:15.737463  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.731034 14578 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:15.745622  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47903.8us
I20260709 13:02:15.747910  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.739089 14738 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.681757 14612 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.752699  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.758275  8823 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:02:15.749831 14578 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:15.756664  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.767906  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:15.753811 14738 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.753233 14571 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:15.754820 14805 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:02:15.779990  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:15.755743 14643 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:15.764428  9444 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:15.780743  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.793653  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.769690 14572 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.770545 14753 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.761571 14812 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.804597  8742 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 2 to 10, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.834133  8747 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:15.817130  8824 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:15.807124  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75094.3us
I20260709 13:02:15.764084 14578 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.806216  8889 raft_consensus.cc:2376] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14. Re-sending same reply.
I20260709 13:02:15.838840  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.834945  8894 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:15.839645  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.841907  8895 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:15.841689 14822 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.840566  8815 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:15.801689 14643 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.852488  9414 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.829926 14572 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:15.845820 14812 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.865587  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:15.865505  8745 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:15.846220 14578 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.869359  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:15.880025  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.872936  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.872082  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.882205  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.845399 14571 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.882234  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:15.877727  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48443.7us
I20260709 13:02:15.879319  8823 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:15.873986 14787 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:02:15.875918  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:15.882812  8815 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:15.877300  8890 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.878546  8891 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:15.853385 14822 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.857107 14643 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:15.881629  8893 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:15.871121 14572 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.867834 14829 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:15.899053  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.901413  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:15.877803 14694 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.898461 14821 raft_consensus.cc:2804] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:15.907650  8889 raft_consensus.cc:1217] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 12.7->[16.8-16.8]   Dedup: 16.8->[]
I20260709 13:02:15.911388  8889 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:15.908767  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:15.907934  8712 leader_election.cc:336] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.132s (SENT)
I20260709 13:02:15.915230  8895 raft_consensus.cc:2368] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
W20260709 13:02:15.916038  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65682.7us
I20260709 13:02:15.916376  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.893983 14650 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:15.912750 14694 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.918120 14612 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.919687  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.920038 14694 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.921701 14804 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:15.922163 14804 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.922557 14804 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:15.925724  8897 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.928212  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.930416  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:15.930799 14571 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:15.931397  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:15.892163 14822 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.932760 14799 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:15.937395 14799 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:15.937603  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.937832 14799 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:15.921049  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.939864  8820 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:15.941120  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.941509  8823 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:15.942440  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.942965  8823 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:15.947867 14821 raft_consensus.cc:697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:15.950969  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1018) took 1057 ms. Trace:
I20260709 13:02:15.951407  9444 rpcz_store.cc:276] 0709 13:02:14.897100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.905627 (+  8527us) service_pool.cc:224] Handling call
0709 13:02:15.950776 (+1045149us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b5400197654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:15.950889 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1910237056}
I20260709 13:02:15.952385 14804 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.953738  8864 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:15.954262 14741 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.955164  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:15.956027 14805 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:15.956772 14496 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:15.957218 14496 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.957644 14496 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.882427  8641 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:15.958709  8641 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 10, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:15.936120  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.961462  8819 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:15.962054  8824 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:15.858096 14753 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.968305  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:15.968935  8897 raft_consensus.cc:2410] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:15.965804  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:15.963464  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67082.6us
I20260709 13:02:15.956602 14703 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:15.956604 14805 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:15.988142  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:15.989113  8897 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:15.991272  8712 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.203s (SENT)
I20260709 13:02:15.991889  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:15.992338  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.912964 14411 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:15.964165 14697 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:15.978538 14571 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:16.002622  8819 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:16.000030  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 325486us
I20260709 13:02:16.004809  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:15.960621  8641 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 12, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 12 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:15.982641 14650 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:16.012179 14481 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:16.013077 14481 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:16.016242  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.018468 14481 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.019686  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.020059 14805 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.032352  8889 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:16.004737 14571 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.003865 14611 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.046864  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:15.949040 14821 consensus_queue.cc:237] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.001809 14528 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.063870  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:16.048622  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.053969  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.056129  8897 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:16.058616  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.059664  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.061196  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.067425  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:16.070240  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.073552  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.104429  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.106926  8898 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.036298 14697 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:16.111158  8740 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:16.111886  8740 raft_consensus.cc:606] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:16.111945  9446 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:16.113078  8893 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:16.112356  8740 raft_consensus.cc:614] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:16.058311 14571 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:16.114847  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.116209  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.118865  9440 raft_consensus.cc:2410] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:16.122256  8741 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:16.119700  8892 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:16.125363  8745 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.120983  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.128736  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.129665  8743 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:16.134487  8748 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:16.135876  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.137682  8896 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:16.139786  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 158774us
I20260709 13:02:16.061889 14822 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.135380  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:16.068496 14611 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.147842  8890 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:16.153612  9415 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:16.129042 14528 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:16.162577  8741 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:16.164155  9446 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:16.142701 14615 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.142885 14580 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.171680  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.172595  8897 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:16.177241  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 142801us
I20260709 13:02:16.158383 14812 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.183420  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:16.175534 14804 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:16.165685 14528 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:16.168417 14703 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.192736  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:16.194238  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.168730 14716 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:16.191155  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.202958  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:16.210827  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.147243 14650 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.190650 14795 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:16.221386  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.214926  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:16.215226  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61726us
I20260709 13:02:16.176146 14580 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.222035  8748 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:16.219535  8745 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:16.224260  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:16.234715  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.220827  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:16.210060 14712 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.004s
I20260709 13:02:16.240233  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:16.177896 14615 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.213763 14350 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:02:16.247864  8740 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:16.204886 14753 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.254402  9442 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:16.257530  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.225368 14795 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.261034  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.262122  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.243530 14411 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:16.266710  8894 raft_consensus.cc:2368] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
W20260709 13:02:16.268612 14838 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:02:16.243336 14741 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:16.269625  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.224602 14520 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:16.283139  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.285856  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.286176 14822 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.286573  8748 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:16.222604 14738 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.261811 14532 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:16.189348  8641 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:16.293219 14528 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:16.295461  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.258678 14796 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:16.265381 14795 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:16.293638  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.340703  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:02:16.308796  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.254989 14839 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:02:16.353790  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.354669  9445 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:16.357640  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.298681 14532 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:16.362887  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 101940us
I20260709 13:02:16.363924  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:16.369518  8740 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:16.293970 14741 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.381430  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:16.367028 14831 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:16.298619 14520 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:16.404431  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.405018  9440 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:16.368017 14532 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:16.405668  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.409071  8740 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:16.336066 14572 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.373733 14611 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.413094  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:16.421267  8822 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:02:16.422158  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:16.428151  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.428833  8895 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:16.429636  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.429718  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:16.431190 14848 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:02:16.432353 14856 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:02:16.433370 14857 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:02:16.433784  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.436308 14520 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:16.437139 14846 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:02:16.441704  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.443261  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.444087  8748 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:16.444986  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:16.445415 14799 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.010s
I20260709 13:02:16.445851 14799 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.446293  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.447960 14799 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.448036 14496 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.409857 14622 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:16.450016 14622 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.450387 14622 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:16.451371 14697 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:16.451793 14697 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:16.415350  8641 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:16.458843 14738 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.459676  8641 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:16.460973  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.462719  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.471025  8895 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.463222  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.471747  8898 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:16.472107  8897 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:16.463595  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.463990  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.465224  8891 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:16.466924  9444 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:16.465705  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.452642 14697 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.462263  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.473976  8896 raft_consensus.cc:2368] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:16.469657  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.464547 14572 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.424014 14580 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:16.476092  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.476202  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.476092  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.495788  8891 raft_consensus.cc:3060] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:16.496470  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.496639  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.497457  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.498085  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.499194  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.500166  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.480715  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.501966 14496 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.496491  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.481945 14615 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:16.463312 14578 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.575s	user 0.005s	sys 0.003s
I20260709 13:02:16.503501  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:16.503873 14578 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.504752  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.501986  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.505991 14578 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.482075  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.506474  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.507357  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:16.501111  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.508555  9442 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:16.510020  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:16.497507  9441 raft_consensus.cc:2410] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:16.483595  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.485409  8895 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
W20260709 13:02:16.474030  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54630.5us
I20260709 13:02:16.405002 14796 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:16.482414 14571 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.516978  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.529518 14697 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:16.522315  8891 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:16.523998  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.524721  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.583280  8895 raft_consensus.cc:2410] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:16.608271  8890 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:16.610991  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.612589  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.613636  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.616986 14622 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.618502  9443 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 19. Ops: []
I20260709 13:02:16.625658 14622 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.626964 14864 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:16.627480 14864 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.461233  8641 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 14, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 14 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:16.630158 14864 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:16.629966  8641 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 12, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:16.631531  8641 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 14, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 14 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:16.522951  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:16.637849 14697 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:16.638664  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 115840us
I20260709 13:02:16.639582  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:16.641060  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:16.643795 14712 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:16.644676 14712 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:16.644518 14532 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:16.646430 14643 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.743s	user 0.008s	sys 0.001s
I20260709 13:02:16.648541  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.502058 14839 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 13:02:16.654111 14796 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:16.657799  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 123031us
I20260709 13:02:16.678807  9445 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:16.679991  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.683861  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.584106 14496 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:02:16.699910  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.675590 14839 consensus_queue.cc:1243] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:16.641193 14795 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.710708  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.675925 14643 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.702961 14799 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.762538  8893 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:16.762423  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84281.9us
I20260709 13:02:16.764582  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.632980  8641 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 14, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 14 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:16.775223  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:16.777647  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 93752.2us
I20260709 13:02:16.777825  9447 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:16.732816 14716 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.788941  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.794234  8741 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:16.779841 14796 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.802275  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:16.802122  8740 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.813516  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.814879 14496 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.819415  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.804725  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:16.838533 14815 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.005s	sys 0.023s
I20260709 13:02:16.817561 14738 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:16.851534  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.852973  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.853801  8897 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:16.854781  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.855358  8897 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.860553  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.862354  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.863758  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.864837  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:16.865949  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.869439  8897 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
W20260709 13:02:16.849548 14815 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.005s	sys 0.024s
W20260709 13:02:16.844269 14868 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:02:16.879626  9440 raft_consensus.cc:1217] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 16.12->[17.13-20.15]   Dedup: 17.13->[18.14-20.15]
I20260709 13:02:16.878479  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:16.879956  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62691.3us
I20260709 13:02:16.847726 14571 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:16.882736  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:16.881156  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.880172  9440 pending_rounds.cc:85] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 13
I20260709 13:02:16.885665  9440 pending_rounds.cc:107] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:02:16.894183  9440 raft_consensus.cc:2889] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:16.897706  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 87394.8us
W20260709 13:02:16.898910  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.899340  8748 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:16.901257  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.902809  8892 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:16.915617  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:16.879386 14606 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.885856 14622 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:16.921134  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:16.928539  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:16.898806 14796 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:16.932469 14837 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.748s	user 0.003s	sys 0.006s
I20260709 13:02:16.932583  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:16.934090 14472 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:16.934789  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:16.934953 14472 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.935688 14472 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:16.935961  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:16.881763 14738 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:16.936012 14472 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:16.978408  9442 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:16.937161  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:16.845842  8641 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 9, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:16.945461  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:16.980957 14571 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:16.921656 14496 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.948530 14859 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.541s	user 0.004s	sys 0.008s
I20260709 13:02:16.938879 14865 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.982192  8895 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:16.949572 14845 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.537s	user 0.004s	sys 0.005s
I20260709 13:02:16.943730  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:17.004307  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 123216us
I20260709 13:02:17.011456  8889 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:16.946492 14795 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:16.947317 14870 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:02:16.978312 14622 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:16.982657  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:16.946430 14836 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.769s	user 0.002s	sys 0.021s
W20260709 13:02:16.983256 14837 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:02:16.946874 14643 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:17.012341  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:17.012388  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:17.019562  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:17.020017  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.942394 14741 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:17.017745  8895 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:17.029341  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 108275us
I20260709 13:02:17.030946  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:17.009655 14496 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:17.032876  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 73524.5us
I20260709 13:02:17.019403 14697 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.026357 14606 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.028254 14795 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.012835 14865 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.048475 14865 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:17.049453 14622 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.055871 14738 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:17.059710 14496 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:17.061178  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1011) took 1329 ms. Trace:
I20260709 13:02:17.030040 14472 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:17.062816 14472 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:17.061650  8816 rpcz_store.cc:276] 0709 13:02:15.731304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.731448 (+   144us) service_pool.cc:224] Handling call
0709 13:02:15.731799 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.732175 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:15.732438 (+   263us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.982205 (+249767us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.982605 (+   400us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.982718 (+   113us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:15.982738 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.982784 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.931921 (+949137us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:17.060820 (+128899us) raft_consensus.cc:1685] finished
0709 13:02:17.060956 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.061100 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8135,"prepare.queue_time_us":700,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":9830,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":18514,"spinlock_wait_cycles":140427136,"thread_start_us":246066,"threads_started":1,"wal-append.queue_time_us":395591}
I20260709 13:02:17.066464  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:17.068348  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:17.070092 14697 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.071331 14697 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:17.071440 14863 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.664s	user 0.000s	sys 0.002s
I20260709 13:02:17.071738 14697 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:17.075322  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2847) took 1009 ms. Trace:
I20260709 13:02:17.075838  8823 rpcz_store.cc:276] 0709 13:02:16.066201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.199618 (+133417us) service_pool.cc:224] Handling call
0709 13:02:16.199899 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.200063 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:16.220649 (+ 20586us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.074660 (+854011us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800461da0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:02:17.074866 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.074928 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:17.074988 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.075079 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.075244 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5014,"prepare.queue_time_us":24552,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":4170,"spinlock_wait_cycles":23470848}
I20260709 13:02:17.080106  8895 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:17.081168  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:17.081954  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:17.085641  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 845468us
W20260709 13:02:17.085700  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 864951us
I20260709 13:02:17.087963  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:17.090950  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.091967  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.097465  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 51336.1us
I20260709 13:02:17.099470  8862 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:16.937490 14841 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:02:17.156730  9447 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:17.114393  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.153143  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:17.129863  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.132593  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.143159  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:17.112838  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:17.121997  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.141378  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 1086) took 1130 ms. Trace:
I20260709 13:02:17.159344  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.118242  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.130430  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.167150  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.124935  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.214654  8897 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:17.223356  8819 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:17.166330  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:17.219401  8824 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.223302  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:17.222996  9444 rpcz_store.cc:276] 0709 13:02:16.010523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.010662 (+   139us) service_pool.cc:224] Handling call
0709 13:02:17.141071 (+1130409us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54001982d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:17.141275 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026760576}
I20260709 13:02:17.224651  8898 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:17.135555  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:17.165589  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:17.142078  9443 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:17.138758  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.115526  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:17.117537  9449 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:17.275897  8889 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:17.103358 14829 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:17.118090  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:17.125926  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:17.089363 14697 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.298445  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.356678  8822 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:17.357259  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 221766us
I20260709 13:02:17.306681  8897 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:17.358085  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:17.280972  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 212720us
W20260709 13:02:17.108533 14873 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
W20260709 13:02:17.137342 14843 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.964s	user 0.017s	sys 0.004s
I20260709 13:02:17.212952 14496 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:17.103220 14611 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:17.282083 14839 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:02:17.298295  8898 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:17.362344  8898 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:17.159551 14532 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:17.299909  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 133654us
I20260709 13:02:17.303361  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:17.232084 14796 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:17.363570  8898 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.355973 14580 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.382021  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:17.383315  9447 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:17.361459  8745 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:17.281896  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 194084us
I20260709 13:02:17.359227 14606 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:17.299971 14825 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:17.278228 14612 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:17.374413  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80537us
I20260709 13:02:17.375139  8898 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.369470 14694 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:02:17.376482 14528 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:17.342398  8641 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 12, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:17.368913 14738 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:17.378494  8743 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:17.378821  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:17.307981 14854 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.774s	user 0.000s	sys 0.002s
W20260709 13:02:17.491544  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 474914us
I20260709 13:02:17.500738  8893 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:17.501417  9443 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:02:17.445981  8896 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
W20260709 13:02:17.476374  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90498.3us
I20260709 13:02:17.402910  8824 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:17.360689  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:17.124068 14833 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.968s	user 0.005s	sys 0.010s
W20260709 13:02:17.383486  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.389940  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:17.512063  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 122231us
I20260709 13:02:17.362654 14829 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.404652  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:17.427675  9442 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
W20260709 13:02:17.137483 14832 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.950s	user 0.003s	sys 0.008s
W20260709 13:02:17.110595 14868 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:02:17.526026  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:17.528855  9445 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:02:17.482879 14805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.459s	user 0.033s	sys 0.070s
I20260709 13:02:17.402007  8819 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
W20260709 13:02:17.162966 14874 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:02:17.502007  9443 pending_rounds.cc:85] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 13
I20260709 13:02:17.534703  9443 pending_rounds.cc:107] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:02:17.535121  9443 raft_consensus.cc:2889] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:17.521950  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 117417us
I20260709 13:02:17.537467  8744 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:17.538864  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:17.533527  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.27187e+06us
I20260709 13:02:17.544137  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2200) took 1104 ms. Trace:
I20260709 13:02:17.544699  8743 rpcz_store.cc:276] 0709 13:02:16.439567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.112429 (+672862us) service_pool.cc:224] Handling call
0709 13:02:17.543964 (+431535us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54000217d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:17.544083 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358339456}
I20260709 13:02:17.522436 14865 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:17.522608 14829 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:17.533161 14805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.509s	user 0.033s	sys 0.074s
I20260709 13:02:17.145816 14828 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.387s
I20260709 13:02:17.537366 14637 raft_consensus.cc:2764] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:17.491438 14697 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:17.598168  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:17.521194 14528 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:17.625584 14528 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.626986 14528 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.625584  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1031) took 1135 ms. Trace:
I20260709 13:02:17.629021  8894 rpcz_store.cc:276] 0709 13:02:16.489815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.498317 (+  8502us) service_pool.cc:224] Handling call
0709 13:02:16.514472 (+ 16155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.538633 (+ 24161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.538681 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.585600 (+ 46919us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.812957 (+227357us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:16.813001 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.813071 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.162141 (+349070us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:17.625240 (+463099us) raft_consensus.cc:1685] finished
0709 13:02:17.625260 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.625519 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9591,"raft.queue_time_us":67692,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":38645,"spinlock_wait_cycles":9929600,"thread_start_us":121232,"threads_started":1,"wal-append.queue_time_us":202669}
I20260709 13:02:17.631902  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:17.135987 14862 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.397s
I20260709 13:02:17.506486 14825 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:17.547241 14353 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:17.601351  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1042) took 1829 ms. Trace:
I20260709 13:02:17.547052 14795 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:17.589733 14650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.001s	sys 0.014s
W20260709 13:02:17.602195  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 241590us
I20260709 13:02:17.603883  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1038) took 1929 ms. Trace:
I20260709 13:02:17.414745 14796 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:17.404071 14580 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.518561 14612 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.527185 14716 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:02:17.532238 14799 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 13:02:17.680294 14808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.013s
W20260709 13:02:17.615123 14833 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 13:02:17.598847 14887 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.716791 14887 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.717592  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1008) took 2036 ms. Trace:
I20260709 13:02:17.719074 14887 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:17.595816 14891 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:17.718004  8821 rpcz_store.cc:276] 0709 13:02:15.681547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.681699 (+   152us) service_pool.cc:224] Handling call
0709 13:02:15.686927 (+  5228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.692273 (+  5346us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:15.692611 (+   338us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.895155 (+202544us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.895638 (+   483us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.895890 (+   252us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.923193 (+ 27303us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b1800352660. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:15.934403 (+ 11210us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.934601 (+   198us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:15.934647 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.973154 (+ 38507us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.123345 (+1150191us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:17.717351 (+594006us) raft_consensus.cc:1685] finished
0709 13:02:17.717374 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.717538 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51265,"prepare.queue_time_us":17209,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":122,"raft.queue_time_us":2890,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":26758,"spinlock_wait_cycles":142082176,"thread_start_us":162995,"threads_started":1,"wal-append.queue_time_us":415481}
I20260709 13:02:17.719740 14891 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:17.720206 14891 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:17.668958  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70884.2us
I20260709 13:02:17.670751 14611 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:17.619843 14622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.011s	sys 0.027s
W20260709 13:02:17.381088 14496 consensus_queue.cc:1175] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b60591217ee24378bb66a86d6d0dda52" responder_term: 17 status { last_received { term: 11 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:17.725220 14828 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:17.677568  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.678494  8749 rpcz_store.cc:276] 0709 13:02:15.674636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.674773 (+   137us) service_pool.cc:224] Handling call
0709 13:02:15.683678 (+  8905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.046907 (+363229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.046939 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.117568 (+ 70629us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.234057 (+116489us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:16.234108 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.236117 (+  2009us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.307220 (+1071103us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:17.593814 (+286594us) raft_consensus.cc:1685] finished
0709 13:02:17.593835 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.603704 (+  9869us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10678,"raft.queue_time_us":40476,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":15962880,"thread_start_us":422549,"threads_started":2,"wal-append.queue_time_us":296239}
I20260709 13:02:17.672999  8742 rpcz_store.cc:276] 0709 13:02:15.767320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.767507 (+   187us) service_pool.cc:224] Handling call
0709 13:02:15.801257 (+ 33750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.804446 (+  3189us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:16.301155 (+496709us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.070772 (+769617us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:17.150815 (+ 80043us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b18004e45a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:02:17.592679 (+441864us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.596180 (+  3501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.596229 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:17.596271 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.596334 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.596437 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3258,"prepare.queue_time_us":377273,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":5378,"raft.queue_time_us":3496,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6886,"spinlock_wait_cycles":36135808,"thread_start_us":483676,"threads_started":2,"wal-append.queue_time_us":105845}
I20260709 13:02:17.678761  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:17.672394 14863 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:02:17.678247 14650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.002s	sys 0.015s
I20260709 13:02:17.538975 14643 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:02:17.624220 14738 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:17.672633 14825 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.674512 14353 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:17.634130 14823 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:02:17.754686  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 76004.1us
W20260709 13:02:17.722615 14808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.006s	sys 0.013s
I20260709 13:02:17.725695 14828 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:17.801805  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.725508 14795 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:02:17.759680 14472 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:17.756920 14716 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:17.811487  9441 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:02:17.817574 14883 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:02:17.818601 14890 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:02:17.820201 14892 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:17.770131 14353 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.827515  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:17.804226  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:17.824375  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:17.828776  8745 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:17.756739 14694 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
W20260709 13:02:17.825259  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.832656  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.815142 14716 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:17.849977 14697 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:17.850169 14886 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.851354  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.852924 14580 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:17.853524 14580 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:17.855136 14520 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.855635 14520 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:17.859715 14716 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.861140 14862 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:17.861632 14862 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:17.862092 14887 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.862732 14887 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.862663 14862 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:17.864729 14697 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:17.865219 14697 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:17.879906  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75787.2us
I20260709 13:02:17.836827 14611 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:17.898537  8822 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:02:17.899008  8822 pending_rounds.cc:85] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 13
I20260709 13:02:17.899277  8822 pending_rounds.cc:107] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:02:17.899556  8822 raft_consensus.cc:2889] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:17.902549 14612 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:17.902747 14899 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.903097 14612 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.903592 14612 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:17.905551  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:17.932432  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:17.822778 14738 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:02:17.678315  8641 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:17.937108  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1032) took 1402 ms. Trace:
I20260709 13:02:17.937803  8823 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:17.900534 14611 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.993498  8891 rpcz_store.cc:276] 0709 13:02:16.582174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.677088 (+ 94914us) service_pool.cc:224] Handling call
0709 13:02:16.700626 (+ 23538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.706508 (+  5882us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:16.768600 (+ 62092us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.148998 (+380398us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b18000adf20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:02:17.936759 (+787761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:17.936810 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:17.936855 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.936937 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.937032 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3084,"prepare.queue_time_us":57819,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":242545,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":11275392,"thread_start_us":784354,"threads_started":2,"wal-append.queue_time_us":1128747}
W20260709 13:02:18.007480  8892 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.008s
W20260709 13:02:18.007944  8892 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.011s	sys 0.008s
W20260709 13:02:17.823344  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59890.6us
I20260709 13:02:18.009322  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:18.037819  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 491997us
W20260709 13:02:18.007283 14741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.010s	sys 0.010s
I20260709 13:02:17.938325 14571 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:18.041882  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 208130us
I20260709 13:02:18.046622  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:18.006161 14611 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:18.049767  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.052507 14796 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:18.053083  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 151245us
I20260709 13:02:18.053567  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2550) took 1277 ms. Trace:
I20260709 13:02:18.057559  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.058076  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:18.028940 14520 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.064688 14823 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:18.065395 14823 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.065855 14823 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:18.074851  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:18.046275 14741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.011s	sys 0.010s
I20260709 13:02:18.053370 14904 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:18.051472 14612 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.053735 14615 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:18.045758 14865 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:18.049573 14532 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:18.087327  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:17.897965 14828 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.076345 14812 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.060395 14796 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:18.057201  8895 rpcz_store.cc:276] 0709 13:02:16.776145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.082290 (+306145us) service_pool.cc:224] Handling call
0709 13:02:17.082660 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.083006 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:17.083421 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.810087 (+726666us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.022152 (+212065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.022220 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:18.053304 (+ 31084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.053381 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.053505 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2019,"prepare.queue_time_us":875,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":82,"raft.queue_time_us":296202,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":11502,"thread_start_us":925671,"threads_started":3,"wal-append.queue_time_us":590251}
I20260709 13:02:18.115546  8820 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:18.116446  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:18.118006  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 184160us
I20260709 13:02:18.110476 14823 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.108023  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:18.122349  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:18.132889  8818 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:18.126391 14829 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.101562 14741 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:02:18.115159 14812 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.098012 14615 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:18.128260 14528 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.005s
I20260709 13:02:18.087935 14904 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.130640  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.112468 14909 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.133981  8815 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:18.132259  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.135347  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:18.135512  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.141851  8712 leader_election.cc:400] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
I20260709 13:02:18.161540  8712 leader_election.cc:403] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.143131 14812 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.122076 14840 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:18.117511 14532 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.165120  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:17.903498 14899 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.165895  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:17.765164 14894 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.395s
I20260709 13:02:18.122344 14796 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.173349  8816 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:18.161342  8823 raft_consensus.cc:2376] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19. Re-sending same reply.
I20260709 13:02:18.153362 14615 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.181751  8818 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:18.153054 14799 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:18.159657 14528 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.229755 14904 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.217257  9444 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
W20260709 13:02:18.232717 14896 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:02:18.162966 14520 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:18.232097 14911 log.cc:927] Time spent T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:18.163319 14909 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.228766 14924 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:18.216566  8819 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:17.850673 14886 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:18.175007 14901 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
W20260709 13:02:18.231143  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 109671us
I20260709 13:02:18.162920 14795 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:18.273842  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 224132us
I20260709 13:02:18.151031 14865 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.217753 14840 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.287353  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.233369 14918 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:18.298317  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 210374us
I20260709 13:02:18.240001 14815 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 13:02:18.311079  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 154947us
I20260709 13:02:18.285775 14528 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.320838  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.327231  8863 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.329116 14716 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.329684 14909 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.204638  8641 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:18.355281  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:18.356050  9443 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.355795  8641 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 11, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 11 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:18.352393 14840 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.357635 14930 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.358091  8863 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:18.358976  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.359166 14840 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.359565 14886 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.359887 14917 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:18.360167  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.360914 14935 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.317086 14741 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:18.361451 14741 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:18.361658 14935 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:18.361898  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.362033 14935 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.362510 14935 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.362321 14741 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.363690 14741 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:18.336184 14906 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.365749 14906 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.366039 14915 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:18.367125 14823 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.367250 14915 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:18.367727 14915 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.369042  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:18.369819  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75713.6us
I20260709 13:02:18.304883 14926 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
I20260709 13:02:18.370851  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.337419 14899 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.372073  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:18.338514 14532 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:18.340338 14893 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.743s	user 0.011s	sys 0.000s
I20260709 13:02:18.348112 14795 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.374053 14795 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.374274  8641 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 12, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:18.341648 14903 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:18.375567  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2212) took 1871 ms. Trace:
I20260709 13:02:18.375707 14903 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.376689 14795 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:18.376639 14903 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.377772 14920 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:18.378126 14903 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.378294 14920 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:18.379645  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1173) took 1911 ms. Trace:
I20260709 13:02:18.380007  9441 rpcz_store.cc:276] 0709 13:02:16.468028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.510983 (+ 42955us) service_pool.cc:224] Handling call
0709 13:02:16.511285 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.812738 (+1301453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.379358 (+566620us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003a2854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:18.379563 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982220032}
W20260709 13:02:18.338354  8787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.018s	sys 0.038s
I20260709 13:02:18.347265 14829 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.342468 14796 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:18.360316 14917 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.381800 14917 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.373237  9443 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:18.379277 14920 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.382160 14938 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.383797 14904 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:18.381086  8787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.018s	sys 0.039s
W20260709 13:02:18.385115  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 785284us
I20260709 13:02:18.385718 14606 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:18.386163 14606 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.386519  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:18.387434 14891 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.003s
I20260709 13:02:18.386970 14606 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.307159 14929 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.388302 14606 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:18.387887 14891 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.389358 14829 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.390470 14891 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.390774  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.388720 14929 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.392159 14622 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:18.392550 14622 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:18.393471 14622 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.394507 14622 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:18.395088 14929 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.395371 14932 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:18.395730 14350 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.395975 14932 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.397686 14796 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:18.398209 14796 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.400377 14697 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.401052 14916 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:18.402427 14932 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:18.403424  8641 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:18.354449 14943 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:18.368615 14915 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.373097  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:18.401104  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80837.9us
I20260709 13:02:18.401082  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:18.409577  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 312814us
W20260709 13:02:18.405480  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 55200.1us
W20260709 13:02:18.428344  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55320.4us
I20260709 13:02:18.429240  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.429893  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:18.436404  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.440919  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.442827  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:18.364781 14611 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:18.577454  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.375984  8747 rpcz_store.cc:276] 0709 13:02:16.504517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.938667 (+434150us) service_pool.cc:224] Handling call
0709 13:02:16.938986 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.939362 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:17.018921 (+ 79559us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.592082 (+573161us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.597831 (+  5749us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.598020 (+   189us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:17.615125 (+ 17105us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.617544 (+  2419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.617629 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.719706 (+102077us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4800080058. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:02:18.339762 (+620056us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:18.375357 (+ 35595us) raft_consensus.cc:1685] finished
0709 13:02:18.375380 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.375498 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70081,"prepare.queue_time_us":74641,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":57741,"raft.run_cpu_time_us":2313,"raft.run_wall_time_us":135928,"spinlock_wait_cycles":20609664,"thread_start_us":572219,"threads_started":2,"wal-append.queue_time_us":511543}
I20260709 13:02:18.407743  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:18.431387  8821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.013s	sys 0.005s
W20260709 13:02:18.580567  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 145292us
W20260709 13:02:18.584764  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 155018us
W20260709 13:02:18.592067  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 191077us
W20260709 13:02:18.588907  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 181244us
I20260709 13:02:18.400429 14796 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:18.427904 14862 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.462172 14886 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.599793  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.600432  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:18.607578  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.617630  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.619081  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:18.604717  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.620373  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:18.621002  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.621863  8742 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:18.598814  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:18.594815  8821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.014s	sys 0.005s
W20260709 13:02:18.612710  9446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.008s	sys 0.057s
I20260709 13:02:18.619719  8745 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:18.434273 14795 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.576843 14841 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:18.580267 14350 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:18.626585  9446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.008s	sys 0.057s
I20260709 13:02:18.576042 14916 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:18.663662  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.627871  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.664315  8747 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:18.663677  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:18.664419  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44149.1us
I20260709 13:02:18.626916  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.627365  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:18.404882 14935 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:18.607877 14353 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.002s
I20260709 13:02:18.658041  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:18.666160  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:18.667757  9441 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:18.592235 14891 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:18.667773  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.434018 14930 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.214s
W20260709 13:02:18.684198  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.608994 14903 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:18.632499 14472 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.000s
I20260709 13:02:18.769353 14912 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:18.769544 14472 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:18.770164 14925 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:18.628085 14908 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.605398 14712 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:18.432837 14943 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.353390 14918 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.626842 14932 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.693812 14894 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:18.664228 14825 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.859s	user 0.007s	sys 0.003s
I20260709 13:02:18.667966 14796 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.695715 14919 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.663381 14697 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:18.676272  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 354574us
I20260709 13:02:18.669227  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1179) took 1788 ms. Trace:
W20260709 13:02:18.759588  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 316855us
I20260709 13:02:18.691078 14924 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:18.668619  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.664208 14795 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:18.754806 14612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.056s	sys 0.090s
I20260709 13:02:18.742343 14353 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.760265 14840 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.576653 14926 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.214s
I20260709 13:02:18.772394  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1047) took 1898 ms. Trace:
I20260709 13:02:18.658370 14839 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.760115  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 96000us
I20260709 13:02:18.774729 14942 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:18.872381 14612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.059s	sys 0.092s
I20260709 13:02:18.874532 14924 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:18.875631 14615 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.000s
I20260709 13:02:18.876084 14615 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.878307 14943 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:18.777872  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 111814us
I20260709 13:02:18.697190 14915 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.883564  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.884671  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:18.881708  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:18.813601  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.812933  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.615571 14886 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:18.888597  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:18.912156  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:18.890870  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1037) took 2242 ms. Trace:
I20260709 13:02:18.775350 14938 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.877902  8898 rpcz_store.cc:276] 0709 13:02:16.874057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.281984 (+407927us) service_pool.cc:224] Handling call
0709 13:02:17.298096 (+ 16112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.696904 (+1398808us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b5400331420. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f644a858 00007f90f64cee9d 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c
0709 13:02:18.742986 (+ 46082us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:18.743034 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.743100 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:18.743174 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.743281 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.772282 (+ 29001us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":878,"raft.queue_time_us":186910,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":124006272}
I20260709 13:02:18.894255  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.897408  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:18.774211  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.883296  9446 rpcz_store.cc:276] 0709 13:02:16.881050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.044827 (+163777us) service_pool.cc:224] Handling call
0709 13:02:17.056780 (+ 11953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.056974 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:17.057273 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.715637 (+658364us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b180035f380. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:18.661080 (+945443us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.668690 (+  7610us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.668766 (+    76us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:18.668831 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.668934 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.669141 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20179,"prepare.queue_time_us":272593,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":3157,"raft.queue_time_us":84898,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":56686,"spinlock_wait_cycles":10154752,"thread_start_us":1590043,"threads_started":3,"wal-append.queue_time_us":547158}
I20260709 13:02:18.842904 14940 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.779978 14925 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:18.861734 14611 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:18.847972 14912 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:18.959113  9441 pending_rounds.cc:85] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 12
W20260709 13:02:18.959394 14948 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:18.961432 14612 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.962073  8787 leader_election.cc:400] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:18.963611  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.029479  8787 leader_election.cc:403] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.964202  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:18.967855 14520 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:19.029942  8748 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:18.964887 14496 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:18.965366  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:18.965608  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:18.965979  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68713.4us
I20260709 13:02:18.839865 14796 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.867550 14697 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:18.667424 14841 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.781847 14712 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:19.061964 14716 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.003s
I20260709 13:02:19.062178 14712 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.062933  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1038) took 1005 ms. Trace:
I20260709 13:02:18.809698 14825 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.785339 14903 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:18.805977 14949 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.957876  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 69607.7us
I20260709 13:02:18.980683  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:18.981551 14950 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.981729  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:18.983071 14947 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:02:18.910364 14943 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:18.883193 14942 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.983575  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.086738  8898 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:18.985157  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:18.954896 14886 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.986356 14795 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.092698 14697 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.967178 14894 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:18.986661  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.095391  8890 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:18.988319  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2858) took 1056 ms. Trace:
I20260709 13:02:18.989495  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:18.989495  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:18.981030  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:18.990474 14916 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.990701  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.095957  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.992647 14865 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:18.998086  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.998917  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:18.999883 14938 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:19.000579 14812 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.822s	user 0.006s	sys 0.002s
I20260709 13:02:19.003033 14828 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:19.004248 14622 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.609s	user 0.004s	sys 0.004s
I20260709 13:02:19.005445 14615 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.008740 14909 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.009156  9448 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:19.012866 14917 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.631s	user 0.003s	sys 0.004s
I20260709 13:02:18.806757 14908 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:18.842674 14799 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.914745 14930 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:19.024327  9441 pending_rounds.cc:107] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:02:19.119392  9441 raft_consensus.cc:2889] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.025827 14912 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:18.976258 14904 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 13:02:19.127655  8895 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:18.775894 14891 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.883942 14933 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 13:02:18.871887 14920 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.130281 14891 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.130306 14920 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.131402  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:19.051940  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.133652  8741 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:19.034219  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:18.867326 14839 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.137924 14795 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:19.058687  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.138545 14795 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:19.138759  8744 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:19.139243 14795 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:19.139912  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.140517  8744 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.971755 14907 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:02:19.059294 14935 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:19.059230  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 97338.5us
I20260709 13:02:19.060710 14918 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.060956 14796 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:19.061371 14741 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.697s	user 0.009s	sys 0.001s
W20260709 13:02:18.968209  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.107115 14580 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:19.149305 14795 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.063349  9444 rpcz_store.cc:276] 0709 13:02:18.057704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.057829 (+   125us) service_pool.cc:224] Handling call
0709 13:02:19.062735 (+1004906us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:19.062873 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255501056}
I20260709 13:02:18.838312 14528 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.887724 14840 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.151700 14841 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.152894 14840 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:18.886533 14919 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.155061 14353 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.099689 14925 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:18.818710 14932 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.158895 14932 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.073055  8893 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:19.104281 14916 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.079447  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.162679  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.163782  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.164710  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.058550 14829 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:19.052270 14496 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:19.086309  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:19.080730 14944 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:02:19.088347 14571 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:02:18.987339 14906 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.096098  9440 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:02:19.099763  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 114719us
I20260709 13:02:19.176985  9415 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:19.178540  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:19.080250 14611 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.100260  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.179014  8744 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:19.080396 14951 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:19.102929  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.103521  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.180637  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.095878  8819 rpcz_store.cc:276] 0709 13:02:17.932329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.932462 (+   133us) service_pool.cc:224] Handling call
0709 13:02:17.932733 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.289863 (+357130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.988097 (+698234us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b540012c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:18.988262 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651779456}
I20260709 13:02:19.088529 14823 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.106432  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.108611  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.110044  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:19.110800  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.111677  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.189069  8823 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:19.189405  8898 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
W20260709 13:02:19.189783  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78167.8us
I20260709 13:02:19.122697 14908 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.123826 14799 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.117568 14520 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:19.125831 14738 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:19.137853  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103716us
I20260709 13:02:19.203023  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:18.916595  8892 rpcz_store.cc:276] 0709 13:02:16.648400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.919152 (+270752us) service_pool.cc:224] Handling call
0709 13:02:16.919445 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.007904 (+ 88459us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:17.008248 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.490956 (+482708us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.716880 (+225924us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b18002e3d80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:17.857917 (+141037us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b18002e3d80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:18.008341 (+150424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.008416 (+    75us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:18.008478 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.890659 (+882181us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b54000492d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:18.890704 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.890817 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10552,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":2390,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":18603,"spinlock_wait_cycles":264659840,"thread_start_us":989621,"threads_started":2,"wal-append.queue_time_us":593402,"wal-append.run_cpu_time_us":2181,"wal-append.run_wall_time_us":1215000}
I20260709 13:02:19.080073 14914 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:19.176329  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.210s (SENT) [suppressed 11 similar messages]
I20260709 13:02:19.191951  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.190841  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:19.169215 14496 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:19.124784 14930 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:19.196322  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.106199 14812 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.212740  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.082769 14950 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.219748  8894 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:19.093966 14949 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.221839 14930 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.224453  8744 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:19.225481  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.181236 14622 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:19.226167  8744 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:19.227056  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.227540 14924 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:19.227882 14606 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.839s	user 0.011s	sys 0.000s
I20260709 13:02:19.228104 14924 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:19.229025 14924 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.182555 14795 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.214293  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.228103  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.102448 14938 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.235590  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.240224  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.238600  8897 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:19.104176 14865 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.243079  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.244782  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:19.244927  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.248708  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.248771  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.249329  8748 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:19.252661  8822 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:19.250665  8789 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:19.255458  8820 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:19.255926  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.256515  8898 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:19.257398  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.258369  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.258997  8898 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 13: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:19.259330  8741 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:19.259757  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.260404  8746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2218) took 1755 ms. Trace:
I20260709 13:02:19.260854  8746 rpcz_store.cc:276] 0709 13:02:17.505030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.505173 (+   143us) service_pool.cc:224] Handling call
0709 13:02:17.528431 (+ 23258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.591864 (+ 63433us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:17.614998 (+ 23134us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.130530 (+515532us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.130689 (+   159us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.145972 (+ 15283us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.153512 (+  7540us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:18.160282 (+  6770us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.160524 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.160619 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.252524 (+1091905us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18003636a0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:19.260198 (+  7674us) raft_consensus.cc:1685] finished
0709 13:02:19.260218 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.260353 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55920,"prepare.queue_time_us":27286,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":1952,"raft.queue_time_us":475491,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":17473,"spinlock_wait_cycles":7467264,"thread_start_us":452338,"threads_started":2,"wal-append.queue_time_us":506677}
I20260709 13:02:19.263144  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.263718  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:19.263849  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.264592  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.266804  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.268764  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.269537  8822 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:19.270383  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.271427  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.196873 14823 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.243971 14650 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:02:19.190639 14580 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:02:19.199074 14954 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:19.169930 14917 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.278573  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.260500  8898 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:19.190806 14611 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.277709  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.280506  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.281802  8748 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:19.282212  8893 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:19.282708  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.241145 14350 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:19.284583  9447 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 12.
I20260709 13:02:19.243784 14805 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.291386  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.288345  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.231622 14932 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:19.289798  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.249032 14606 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:19.290647  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.188136 14951 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.296993  8741 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:19.297586  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2855) took 1574 ms. Trace:
I20260709 13:02:19.297863  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.298523  8741 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:19.297963  8821 rpcz_store.cc:276] 0709 13:02:17.723685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.723979 (+   294us) service_pool.cc:224] Handling call
0709 13:02:17.760794 (+ 36815us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.806265 (+ 45471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.806308 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.826289 (+ 19981us) log.cc:844] Serialized 52 byte log entry
0709 13:02:18.626144 (+799855us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:18.626201 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.626297 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.297404 (+671107us) raft_consensus.cc:1685] finished
0709 13:02:19.297421 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.297538 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39653,"raft.queue_time_us":39828,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5942,"thread_start_us":724991,"threads_started":1,"wal-append.queue_time_us":875551}
I20260709 13:02:19.302045  8815 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:19.216092 14520 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.217012 14738 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.323534  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:19.306655  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.315351  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.307505  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:19.327286  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.327862  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.327286  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:19.329453  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:19.330225  8748 raft_consensus.cc:2376] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15. Re-sending same reply.
I20260709 13:02:19.331264  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.332055  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.309307  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.211534 14903 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.294682 14899 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:19.349709  8748 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:19.290473 14637 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:19.243462 14914 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.343362  8749 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.340296  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:19.345716  8896 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:19.336122  8747 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:19.347802  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:19.294153 14932 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:19.340265  8892 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 12.
I20260709 13:02:19.337512  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:19.351195  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.351513  8748 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:19.294910 14821 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:19.311923 14906 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:19.369010  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:19.307734 14954 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:19.370870  8748 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:19.378377  9444 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:19.333750 14795 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:19.334234 14650 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:19.338685 14887 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.383194  8748 raft_consensus.cc:1275] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:19.338697 14915 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:19.391541  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:19.392535  8889 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.349989 14899 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.349260 14606 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.391170  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.392884  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.430174  8748 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:19.430508  8898 raft_consensus.cc:2410] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:02:19.360235 14796 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.293434 14350 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.374833 14923 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.374580 14821 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:19.435068  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:19.439677  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.380679 14637 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:19.369895 14932 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.446107 14919 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.450024 14916 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:19.462903  8748 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:19.464076  8748 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:19.450757 14825 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.455403 14823 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.456498 14520 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.457599 14912 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.472474  8748 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:19.347884 14961 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:02:19.394228 14915 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:19.480538  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 213828us
I20260709 13:02:19.482744  8889 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:19.435539 14904 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:19.387398 14924 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:19.506268  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.483898 14915 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:19.498138 14912 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:19.509579 14812 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.514070 14912 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.516256 14912 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:19.522002 14962 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 13:02:19.523016 14964 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:19.526712 14966 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:19.494578 14924 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:19.483371  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.435997 14899 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:19.462265 14650 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.462355 14923 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.511987 14904 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:19.525056 14938 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.530827 14799 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.547156 14694 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:19.552932 14963 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:02:19.568183 14940 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.001s
I20260709 13:02:19.555406 14951 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.555856 14350 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.649367  8818 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:19.558053 14865 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.549499 14917 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.558635 14914 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.560982  9449 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:19.561465 14738 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.561591  8892 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:19.548285 14650 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.561918  8889 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:19.562383  8893 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:19.541767 14924 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:19.544468  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61181.3us
I20260709 13:02:19.681097  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:19.394511 14954 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.545580 14611 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.547780 14637 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.548777 14930 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:19.545181 14821 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:19.568846 14612 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 13:02:19.644987  8746 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:02:19.704077  8746 pending_rounds.cc:85] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:19.704368  8746 pending_rounds.cc:107] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:02:19.704612  8746 raft_consensus.cc:2889] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:19.642556 14712 consensus_meta.cc:287] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.003s
I20260709 13:02:19.637636 14694 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.682945  8818 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:02:19.684237  8896 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:19.685806  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:19.553654 14967 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:02:19.649879 14940 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:19.700924  8889 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:19.654567 14936 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.710183  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.710953  8822 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:19.721884  8821 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:19.725018  8749 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:19.727336  8889 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:19.728400  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:19.729126  8889 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:19.741783 14908 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.759042  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.698601 14825 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.713933 14612 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:19.714120 14694 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:19.789882  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.804700  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:19.805723  8896 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.808000  8818 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:19.812193  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:19.761979 14350 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.763703 14829 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:19.820170  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.709303 14712 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.703742 14930 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.802264 14821 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:19.828591  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.834746  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:19.709599 14904 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.824728 14829 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.788887 14917 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:02:19.845774 14917 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:19.846163 14917 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:19.846997 14917 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.847178 14915 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.848243 14917 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.785385 14799 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:02:19.853389  8740 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:19.762492 14936 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.859104 14829 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.862977 14930 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.864225 14932 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.866508 14932 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:19.868072 14496 consensus_meta.cc:287] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.004s
I20260709 13:02:19.868216 14924 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:19.868762 14496 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:19.890139  8818 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:19.871878 14496 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.896054  8898 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: [16.16-16.16]
I20260709 13:02:19.891458  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.907833  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.912602  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:19.915251  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:19.899370  8896 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:19.919682  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:19.907029 14894 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.004s
I20260709 13:02:19.932265  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2506) took 1036 ms. Trace:
I20260709 13:02:19.907189 14923 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:19.933888  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:19.949708  8890 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:19.989912  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.926151 14496 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.955714  8896 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:19.936489 14894 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:20.004173  8895 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:19.943869 14903 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:19.992815 14650 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.012921 14924 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:20.018127 14959 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.810s	user 0.005s	sys 0.001s
I20260709 13:02:20.020092 14712 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.021600 14942 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:19.956022 14915 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.010735 14903 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:20.022843 14903 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:20.024410 14886 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.927s	user 0.008s	sys 0.000s
I20260709 13:02:20.024837 14886 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:20.020946 14909 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.902s	user 0.011s	sys 0.000s
I20260709 13:02:20.035705  8898 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:20.037668 14935 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.038269 14935 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.039038  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2867) took 1440 ms. Trace:
I20260709 13:02:20.039476  8820 rpcz_store.cc:276] 0709 13:02:18.598426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.598568 (+   142us) service_pool.cc:224] Handling call
0709 13:02:20.038827 (+1440259us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54003e35d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.038976 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448943360}
I20260709 13:02:20.041512  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "318340b5bf3b4640972d271299414f94"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:20.003228 14496 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.042096  8747 raft_consensus.cc:606] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.042429  8747 raft_consensus.cc:614] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:20.043605 14496 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:20.045279  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:20.047474  8744 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:20.049046 14935 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.065632  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.057616  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.070005  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.066398  8815 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:20.074360  8895 pending_rounds.cc:85] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 11
I20260709 13:02:20.074715  8895 pending_rounds.cc:107] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:20.074976  8895 raft_consensus.cc:2889] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.073057  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.078953  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.080044  9414 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.080982  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.081624 14865 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:20.083853 14899 consensus_meta.cc:287] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.001s
I20260709 13:02:20.084311 14899 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.085387 14862 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:20.086174 14923 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:20.086592 14923 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:20.086974 14923 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:20.087293 14936 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.088027  8819 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:20.089536 14906 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:20.091845  8745 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:20.092499  8747 raft_consensus.cc:606] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.093219  8747 raft_consensus.cc:614] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:20.094710  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.096112  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.090764 14650 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:20.099732  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.100833  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:20.102371 14968 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.560s	user 0.000s	sys 0.005s
I20260709 13:02:20.101672  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.106197  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:20.102630 14936 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:20.105473  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.091171 14917 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:20.113763  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2497) took 1220 ms. Trace:
I20260709 13:02:20.114223  8742 rpcz_store.cc:276] 0709 13:02:18.893206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.893333 (+   127us) service_pool.cc:224] Handling call
0709 13:02:18.970113 (+ 76780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.186959 (+216846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.186994 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:19.206129 (+ 19135us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.398684 (+192555us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:19.398722 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.398792 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.055648 (+656856us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:20.110251 (+ 54603us) raft_consensus.cc:1685] finished
0709 13:02:20.110274 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.113688 (+  3414us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29156,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":4103936,"thread_start_us":302181,"threads_started":2,"wal-append.queue_time_us":248453}
I20260709 13:02:20.103205 14906 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.105412 14938 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.105260 14961 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:20.105033 14862 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:20.134605  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.113826 14650 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:20.142233  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 1109) took 1011 ms. Trace:
I20260709 13:02:20.142848  9447 rpcz_store.cc:276] 0709 13:02:19.130963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.131145 (+   182us) service_pool.cc:224] Handling call
0709 13:02:20.141989 (+1010844us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400634554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.142155 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350873856}
I20260709 13:02:20.129006 14712 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:20.166019  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1053) took 1093 ms. Trace:
I20260709 13:02:20.126335 14906 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.180605  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.151589 14952 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:20.152016  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.145241 14954 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.128783 14917 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:20.154510  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.191035  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.125568 14865 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:02:20.198988  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 82009.6us
I20260709 13:02:20.137274 14938 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.158483 14950 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.161542  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.163846 14903 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.207646  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.147037 14957 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:20.166512  8891 rpcz_store.cc:276] 0709 13:02:19.072482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.074332 (+  1850us) service_pool.cc:224] Handling call
0709 13:02:19.305701 (+231369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.305923 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.305960 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.306014 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:19.306065 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.306146 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.157928 (+851782us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400048dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.165955 (+  8027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498010880}
I20260709 13:02:20.181620 14839 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:20.149637 14985 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:02:20.186311 14906 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.209012  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.210152  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.211133  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.212147  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.212831  8891 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:20.179227  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.213631 14520 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.134764 14961 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:20.214282 14520 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.215206 14961 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:20.216482  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.156097 14694 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.157334  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.180634 14712 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.205513  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.237725  8897 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:20.205791  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.239207  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:20.213573  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:20.240073  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 82840.6us
I20260709 13:02:20.240839  8891 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:20.241122  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.242810  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.243711  8897 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:20.244772  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:20.217262  8820 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:20.222895  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:20.225236  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:20.228617  8819 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:20.248963  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.229370  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.221764  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.246817  8897 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.226675  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.145545 14936 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:02:20.186993 14650 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:20.186380 14952 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:20.255379  8889 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:20.257578  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2570) took 1063 ms. Trace:
I20260709 13:02:20.257972  8896 rpcz_store.cc:276] 0709 13:02:19.193901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.998936 (+805035us) service_pool.cc:224] Handling call
0709 13:02:19.999414 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.000628 (+  1214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.000670 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.007003 (+  6333us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.127511 (+120508us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:20.127568 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.127661 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.257355 (+129694us) raft_consensus.cc:1685] finished
0709 13:02:20.257393 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.257521 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277,"raft.queue_time_us":3498,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4845,"thread_start_us":34781,"threads_started":1,"wal-append.queue_time_us":136372}
I20260709 13:02:20.261214  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.192823 14969 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.201174 14738 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.265980  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:20.263164  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.267817 14350 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.227057 14821 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.205045 14353 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.265564  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.253140 14815 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:20.217254 14520 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.241675 14950 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.250921 14712 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:20.279590  8891 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:20.270426  8824 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:20.292135  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1056) took 1216 ms. Trace:
I20260709 13:02:20.292553  8893 rpcz_store.cc:276] 0709 13:02:19.075288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.443101 (+367813us) service_pool.cc:224] Handling call
0709 13:02:19.562226 (+119125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.678213 (+115987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.291922 (+613709us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540019e1d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.292078 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273133952}
I20260709 13:02:20.295217  8818 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:20.299000  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2496) took 1406 ms. Trace:
I20260709 13:02:20.299578  8745 rpcz_store.cc:276] 0709 13:02:18.892951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.893060 (+   109us) service_pool.cc:224] Handling call
0709 13:02:18.893364 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.092382 (+1199018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.298673 (+206291us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004475d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:20.298921 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756087680}
I20260709 13:02:20.292930  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.294335  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.313566  8889 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:02:20.313675  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 123363us
I20260709 13:02:20.315822  8820 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:20.271039 14886 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.277379 14940 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:20.319205  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.321033 14350 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:20.342542  8890 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:20.278306 14823 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:20.329658 14520 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.285235 14949 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:19.932843  8741 rpcz_store.cc:276] 0709 13:02:18.896045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.325426 (+429381us) service_pool.cc:224] Handling call
0709 13:02:19.325688 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.325863 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:19.433146 (+107283us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.931660 (+498514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.931803 (+   143us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:19.931891 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.932027 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.932193 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131213,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":6327,"thread_start_us":525447,"threads_started":2}
I20260709 13:02:20.332201 14950 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:20.335650 14987 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.185s	user 0.003s	sys 0.000s
I20260709 13:02:20.341168  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:20.321259  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.387259  8897 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:20.343314  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2500) took 1449 ms. Trace:
I20260709 13:02:20.373852  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:20.390869  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 129733us
I20260709 13:02:20.276885 14738 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.399531  9447 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:20.389909  8740 rpcz_store.cc:276] 0709 13:02:18.894162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.894299 (+   137us) service_pool.cc:224] Handling call
0709 13:02:18.966022 (+ 71723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.859590 (+893568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.343120 (+483530us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b5400021cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:20.343265 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040237312}
I20260709 13:02:20.323529 14796 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:20.399854  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.318207 14899 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.271903 14969 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.413326  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:20.283903 14821 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.413479  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.286895 14353 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.419176  8894 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:20.342787 14350 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.388887 14940 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:20.370745 14520 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:20.421322  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.444057  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.421166  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.351161 14992 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.388293 14957 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:20.339485 14812 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:02:20.339241 14912 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:02:20.414678 14862 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.413393 14936 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.415042 14796 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.445951  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.445937  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.446906  8817 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:20.449308  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 133438us
I20260709 13:02:20.450094  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.444916  8818 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:20.448637  8815 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
W20260709 13:02:20.445513  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 244299us
W20260709 13:02:20.421134  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 79512.3us
I20260709 13:02:20.439169 14350 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:20.443990 14920 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.446260 14520 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.471161 14936 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.503796  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:20.457659 14796 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:20.569820  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2877) took 1395 ms. Trace:
W20260709 13:02:20.599476  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 149473us
I20260709 13:02:20.578706  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:20.411037 14865 consensus_queue.cc:1048] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:02:20.414042 14650 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.344340 14952 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.317463 14917 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.439651 14996 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 13:02:20.574158 14815 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:20.460342  9446 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 16. (term mismatch)
I20260709 13:02:20.597972  8822 rpcz_store.cc:276] 0709 13:02:19.174260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.285465 (+111205us) service_pool.cc:224] Handling call
0709 13:02:20.562517 (+1277052us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400083ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.569748 (+  7231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864711936}
I20260709 13:02:20.454195 14992 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:20.576085 14825 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.774s	user 0.008s	sys 0.011s
W20260709 13:02:20.569820  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.578707 14995 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.119s	user 0.000s	sys 0.003s
I20260709 13:02:20.609638  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:20.611717 14815 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:20.609658  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 335106us
I20260709 13:02:20.585563 15004 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.611111  9446 pending_rounds.cc:85] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 15
W20260709 13:02:20.597801 14991 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.196s	user 0.000s	sys 0.004s
I20260709 13:02:20.633088  9446 pending_rounds.cc:107] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 16.16
I20260709 13:02:20.633885  9446 raft_consensus.cc:2889] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.601261 14894 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.620700 14938 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.622717 14694 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.607963 14932 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.580336 14738 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.624195 14903 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.591161 14998 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.466434 14862 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.611991 14961 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.630054  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.631187  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2228) took 1517 ms. Trace:
I20260709 13:02:20.629456  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.587698 14353 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.650714  8744 rpcz_store.cc:276] 0709 13:02:19.113376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.325729 (+212353us) service_pool.cc:224] Handling call
0709 13:02:20.047284 (+721555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.047865 (+   581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.631036 (+583171us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54001d8b50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:20.631138 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609038592}
I20260709 13:02:20.607869 14920 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:20.628226 14825 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:20.623682 14936 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:20.633735 14942 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:20.628381 14886 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:20.626293 14909 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.633543 14923 consensus_meta.cc:287] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 13:02:20.583177 14906 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.638922 14952 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:20.620007 14999 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:02:20.670840 14999 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:20.583392 14940 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.654841  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.587652 14957 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.633597 14907 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:02:20.662487  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61723.6us
I20260709 13:02:20.650444 14821 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.674309  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.658236 14886 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.711984 14886 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.712209 14906 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:20.708866 14825 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:02:20.715071 14825 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:20.715442 14825 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:20.716223 14825 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.655877 14938 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:20.680311  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.724442 14938 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:20.724891 14938 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:20.724967  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44731.8us
I20260709 13:02:20.677501 14815 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.726138  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:20.726543 14815 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.727313  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.657020 14942 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:20.679140 14353 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:02:20.681932 14999 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:20.743028  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:20.743901  8896 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:20.688339  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 87868.3us
I20260709 13:02:20.733727 14996 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.678987 14920 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:20.694854 14993 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.536s	user 0.002s	sys 0.000s
I20260709 13:02:20.666565 15003 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:20.698103 14961 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.769997 14961 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.769673 15003 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.772552 15003 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.699479 14862 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.773833 14862 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:20.774273 14862 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:20.776113  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.700826 14992 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.703500 14994 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:20.776772 14994 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:20.777668 14994 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.779117 14994 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.640702 15004 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.656068 15006 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:20.668922 14923 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.781747 15006 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:20.782723 15006 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.783617 14923 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.642796 14894 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.664074 14909 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.785472  9449 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:20.646968 14932 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.717703 14935 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:20.674192 14615 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:02:20.719324 15000 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:20.737226  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.795266  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.739372  8891 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:20.667490 14936 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:02:20.742050 14738 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.742916  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:20.743688 14999 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:20.693670 15009 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:20.733912  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 134586us
I20260709 13:02:20.747567  8894 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:20.663028 14969 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.757704  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:20.807070  8898 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:20.807664  8822 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:20.766928 14712 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.730986 14520 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.775429 14862 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.652741 14998 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.701715 14950 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.744645 14996 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.788401  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.825667  8824 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:20.788915  8818 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:20.789085  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.733631 14942 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.730834 14908 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:20.791303  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.828758  8820 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:20.731024 14997 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.733732 14903 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.806452  8816 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:02:20.810473  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2556) took 1652 ms. Trace:
I20260709 13:02:20.793857 15004 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.791250 14935 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:20.773126 15003 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:02:20.828864  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.840919  8892 rpcz_store.cc:276] 0709 13:02:19.158154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.676078 (+517924us) service_pool.cc:224] Handling call
0709 13:02:19.676462 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.857850 (+181388us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:19.877368 (+ 19518us) log.cc:844] Serialized 14 byte log entry
0709 13:02:19.942497 (+ 65129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:19.942560 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:19.942766 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.342571 (+399805us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b180038cda0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419
0709 13:02:20.810177 (+467606us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540045a6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:20.810251 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.810402 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2535,"prepare.queue_time_us":19973,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":770,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2633,"spinlock_wait_cycles":440807168,"thread_start_us":63939,"threads_started":1,"wal-append.queue_time_us":160178}
I20260709 13:02:20.829555 14996 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.863659  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1063) took 1520 ms. Trace:
I20260709 13:02:20.864184  8895 rpcz_store.cc:276] 0709 13:02:19.342972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.029784 (+686812us) service_pool.cc:224] Handling call
0709 13:02:20.074171 (+ 44387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.075492 (+  1321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:20.075524 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:20.075957 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:02:20.127810 (+ 51853us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:20.127841 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.127907 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.694258 (+566351us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:20.851402 (+157144us) raft_consensus.cc:1685] finished
0709 13:02:20.851434 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.863561 (+ 12127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54921088,"thread_start_us":19012,"threads_started":1,"wal-append.queue_time_us":61560}
I20260709 13:02:20.810516 14738 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.846315 14923 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:02:20.843634 14615 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.831312 14825 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:20.861418  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 270134us
I20260709 13:02:20.859782  8898 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:20.840530 14894 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.844937 14961 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.844538 14862 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:20.873649  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 192020us
I20260709 13:02:20.847909 14571 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:20.839582 14904 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 13:02:20.885550  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 140955us
I20260709 13:02:20.844969 14950 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:02:20.845005 14908 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.898138 14894 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:20.899048 15014 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.906911 14909 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:20.908577  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:20.847332 14903 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.914669  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.861617 14796 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:20.920940  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 59992.6us
I20260709 13:02:20.953500  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:20.947700  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:20.948305  8892 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:20.933347  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.967801  8895 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:20.971366  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:20.971763  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91847.1us
I20260709 13:02:20.972833  9415 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:20.973114  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 245882us
I20260709 13:02:20.974224  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:20.920841 15014 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.881930 14615 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.978789  8889 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:02:20.972302 15015 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:02:20.970541 14712 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:20.969542 14997 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.986403  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:20.992306  9449 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:02:20.984277  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:20.917539 14571 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.966674 14938 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:20.994778 15021 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:02:20.982764 14909 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:20.991165 14823 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:02:20.990679 14992 raft_consensus.cc:2764] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:02:20.926891 14894 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:20.983222 14950 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:02:21.048384  9443 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:02:21.048924  9443 pending_rounds.cc:85] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 12
I20260709 13:02:21.049259  9443 pending_rounds.cc:107] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:02:21.049594  9443 raft_consensus.cc:2889] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.988739 14839 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:21.014364 14738 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.014740 14903 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.015041 14932 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.017830 14970 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:20.989377 15014 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.021984  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.022744  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.023778  9448 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:21.023845  9444 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:21.024609  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.026342 14998 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.007437 14350 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:21.073385  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.073612 14350 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.032220  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.074487  9446 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.062269 14612 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:20.973966 14940 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:21.077764 14612 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.078321 14940 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.079973 14612 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.080300 14940 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.048053 14969 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:21.053843  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.081152  9442 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:21.073637 14615 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:20.992045 15006 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:02:21.081962 14615 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:21.082309 14615 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:21.082475 14812 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:21.082861 14812 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:21.061558  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67186.8us
I20260709 13:02:21.033031 15016 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:21.083155 14615 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.083714  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.084482  8898 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:21.041569 15017 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:21.066411 14970 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:21.085417 15017 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.085901 14970 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.087203 14894 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.087657 15017 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.069151  9441 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:21.016757 14886 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.090811 14571 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.090956 14886 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.091197  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:21.082976  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.083766 14812 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.093910  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.091749  9440 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:21.094213  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.043419 14909 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.102439 14909 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:21.103713 14886 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:21.104256 14886 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:21.104769  8890 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:21.096709  8895 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:21.106303 14904 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:21.106832 14904 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.107977 14908 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.108480 14908 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.108799 14904 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.093050  8787 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:21.110421 14940 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:21.110805 14940 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.111119 14940 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.111397 14935 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.112277 14938 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.112736 14935 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:21.113874 14812 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:21.115888 14920 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:21.116707 15025 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.101694 14615 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.117636 15026 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:21.118053 15026 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.118387 15026 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.120118 14912 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.120609 14912 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.105149 14886 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.056727 14839 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.127000  9446 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:21.032820 14961 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:02:21.040038 14694 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:02:21.128384  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.137185  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.138121  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:21.138428  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.131439  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.138746  8747 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:21.040670 14712 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:21.133106  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.141448  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:21.141639  8896 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:21.133769  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.142812  8898 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:21.143535  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:21.144465  8747 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:21.126369  8787 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:21.135219  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.090855 15016 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:21.139333  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.140242  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.140761  8893 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:21.108907 14908 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.149838  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:21.150094  9447 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:21.139197  8890 raft_consensus.cc:2376] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15. Re-sending same reply.
I20260709 13:02:21.150519  8741 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:21.147032  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.147533  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.148468  8897 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:21.136670  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.147137  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:21.153712  8891 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:21.135483 14839 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:21.152935  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:21.157811  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.156142  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.158768  8898 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:21.117213 15025 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.162557 15025 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.132941 14612 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:02:21.166451 14912 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.166925 14908 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.155640 14823 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:21.176476 15023 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.167s	user 0.003s	sys 0.000s
I20260709 13:02:21.176649 14823 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:21.177057 14823 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.158195 14829 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:21.178711  8892 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:21.190641  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.197978  8741 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:21.208848  8897 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:21.155046  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:21.182412  9414 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:21.214461  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 59496us
I20260709 13:02:21.215747  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:21.193615  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 207315us
I20260709 13:02:21.227075  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.189826 14829 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:21.235041  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.225792 15025 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:21.220693 14992 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:21.237268  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.235980  8821 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:21.239252  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.240454  8820 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:21.232950  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:21.243345  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:21.244374  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:21.253777  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.259698  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.260720  9446 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:21.262044  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:21.263495 14862 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:21.263893 14862 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:21.264078  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:21.264320 14862 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:21.265465 14936 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:21.265841 14936 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.266160 14936 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:21.271997 14862 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.273357 14712 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:21.273859 14712 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.274278 14712 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:21.274192 14936 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.276878 14936 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.278003  8898 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:21.284423 15014 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.278636  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:21.281061 14904 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.281664 14908 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.274677 14862 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:21.240962 15025 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.286254  8741 pending_rounds.cc:85] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 11
I20260709 13:02:21.286523  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.310657  8741 pending_rounds.cc:107] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:02:21.310948  8741 raft_consensus.cc:2889] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:21.311460  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.312433  8890 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:21.288411  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:21.236960 14829 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.291078  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.294524  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:21.323450  8824 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:21.296868  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.325138  9442 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:21.236552 15030 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:21.299479  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.302059  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.303725  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.303823  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.283394  8742 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:21.305955  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.308424  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.240172 14992 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.326196  8898 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:21.336664  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.337026  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.337246  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:21.338073  8895 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:21.320921  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:21.320911  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.341609  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:21.343262  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:21.327356  8817 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:21.327345  8821 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:21.286878 15021 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:21.342087  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.346411  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.344761  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:21.347078  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:21.321228  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 228677us
W20260709 13:02:21.349222  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.348174  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.351902  8898 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:21.347713  9448 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
W20260709 13:02:21.346041 14989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.009s	sys 0.007s
W20260709 13:02:21.364506  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.321156 15025 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:21.378639  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:21.364771  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 76487.5us
W20260709 13:02:21.342516 15022 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:21.377547  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:21.378186  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:21.342478 14992 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:21.342341 14949 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:21.414834  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.405987  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:21.412891  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.385243  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 405210us
I20260709 13:02:21.417625  9443 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:21.418294  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.343302 14961 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:21.407786  8742 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:21.383704  8824 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:21.303897 14904 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.304206 14912 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
W20260709 13:02:21.377996 14989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.009s	sys 0.007s
I20260709 13:02:21.427574  8898 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:21.378090 14650 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:21.358224 14611 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:21.356273 14571 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:21.307071 14908 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.458441  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:21.463349  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58795.5us
I20260709 13:02:21.388181 14353 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:21.426398 14942 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.001s
I20260709 13:02:21.470636  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:21.468961  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 66497.8us
W20260709 13:02:21.510512  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.475709  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:21.426333 15030 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:21.480713  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.456367 14952 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:21.459517 14957 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:21.355887 14938 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 13:02:21.466511 14954 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:21.522418  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 64092.6us
I20260709 13:02:21.468886 14611 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:21.462690 14936 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.527963  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 57379.4us
I20260709 13:02:21.402835 14932 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
W20260709 13:02:21.530268  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54618.6us
W20260709 13:02:21.523860  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.456722 14694 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:02:21.536955  8889 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:21.461050 14943 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:02:21.474251 14920 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:21.476742 14353 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:21.534415 14996 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.659s	user 0.005s	sys 0.005s
I20260709 13:02:21.538861 14812 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:21.583945  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 165770us
I20260709 13:02:21.574815  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:21.536005 14954 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:21.551090 15014 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.548951  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:21.471628 14894 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:02:21.531659 15016 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:21.521582 14945 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:02:21.548575 14938 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:21.594182  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:21.583554  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 152035us
I20260709 13:02:21.586529  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.594938  9446 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:21.595474  9442 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:21.545781 15031 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:21.590350  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.592645  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.573094 14950 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:21.593703  9443 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:21.583783 14812 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.526736 15033 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:02:21.585340  8715 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.527189 14942 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.582285  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.602931  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.573921 15005 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:02:21.614449  8824 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:21.597407 14615 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:21.603291  8823 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:21.590516 15032 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:02:21.535600 14957 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.603734  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.609863  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.583942 14920 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.552281 14936 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.618949  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.593091 15016 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:21.607813 14997 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.583s	user 0.000s	sys 0.010s
I20260709 13:02:21.626220  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.607328 15030 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.625461 15014 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:02:21.596402 14862 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:21.627707  8822 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:21.606832 14812 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:21.669446 14997 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:21.670452  8747 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:21.642095  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.585119 14996 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.671531  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.777743  8823 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:21.641439  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.673157  8815 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:21.673539  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.691268  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:21.778436  8820 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.604746 14350 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:21.771168  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 188983us
I20260709 13:02:21.782721  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:21.648444 15039 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:02:21.634445 14909 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.531s	user 0.004s	sys 0.005s
I20260709 13:02:21.614820 15031 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:21.838276  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.646776 15028 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:21.634523 14615 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:21.647130 15037 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:21.776029  8818 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:21.778731  8824 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
W20260709 13:02:21.768181  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.846457  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.847141  8818 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:21.665333 15051 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:21.657279 15035 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:21.655267 15042 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:02:21.647716 15045 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:21.837126  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.842787 14997 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:21.616150 15021 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:21.647117 15004 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:21.599157 14611 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.776731 14862 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:21.863068 14712 consensus_meta.cc:287] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 13:02:21.666476 14938 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.171s
W20260709 13:02:21.836727 15026 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.000s
I20260709 13:02:21.691975 15016 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:21.848181  8817 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:02:21.925791  8817 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:21.925658 15026 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.639475 14839 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.928160 14839 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.640218 14829 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:21.843796 14940 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.732s	user 0.014s	sys 0.008s
I20260709 13:02:21.857869 15050 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:21.929566 15048 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.929419 15050 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.860422 14520 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.930469 14829 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.930189 15048 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.930936  8816 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:21.861976 14908 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.929198 14940 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.849349 15051 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:21.839097 14571 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:21.933163 15051 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.933318 14571 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.934120 14940 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.865608  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:21.843478 15028 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:21.934903 14615 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.935500 14571 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.935603 15028 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.936872 15028 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.866439 14942 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.866761 15027 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
W20260709 13:02:21.868871 15043 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.207s	user 0.000s	sys 0.002s
I20260709 13:02:21.837196  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:21.938532 14996 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:21.861527 15040 log.cc:927] Time spent T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 13:02:21.775702 14954 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.939374  8815 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:21.939643  8817 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:02:21.930272  8744 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:21.853700  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:21.916127 15031 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:21.928090 15026 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.928782 15034 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:21.962128  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 96602.3us
I20260709 13:02:21.931687 15050 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.841809 14909 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.963595  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.942378  8822 raft_consensus.cc:2410] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:02:21.862588 14970 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:21.964737  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:21.964936 14970 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:21.929639  8818 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:21.965476  8890 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:21.945243  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.917639 15037 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:21.913952 15004 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:21.932447 15048 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.864985 14998 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:21.964829 14909 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:21.969313 14970 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.970196 14520 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:21.970908 14353 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:21.972793 14920 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.922713 14862 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.006937 14353 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.973443 14954 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.009382  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.011564  9444 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:02:21.955029 14611 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.980789 15034 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:21.974661 14957 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:21.986939 15021 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:21.989966 14906 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:22.002416 14992 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.000s
I20260709 13:02:22.005625 14970 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.006170 14520 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:21.943707 15017 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:21.965483 14942 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:22.035765  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.045454  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 321100us
I20260709 13:02:21.984459 15026 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.942754 14712 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:22.052223  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 266381us
W20260709 13:02:22.041857 14961 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.010s
W20260709 13:02:22.055290  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 90626.4us
I20260709 13:02:22.055362  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.060249  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:22.062151  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.032265 15005 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.028784 14906 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:22.017494 15034 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.038820 14520 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:22.075520  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.054934 14935 consensus_meta.cc:287] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.942s	user 0.013s	sys 0.000s
W20260709 13:02:22.064751 14961 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.013s	sys 0.010s
I20260709 13:02:22.078998  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.086009  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.084052  8820 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:22.089461  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.065376 14996 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:22.039713 14992 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.076146 15013 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:22.076128 14920 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.052610 15017 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.074023 14936 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.085435 14943 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.091655 14894 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:22.085700 14961 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:22.067536 15026 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.091346 14950 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.132176  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.129801  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:22.115415 15025 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.002s
I20260709 13:02:22.066592 15031 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.161742  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.077683 15005 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.130370  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.112612 15013 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:22.169845  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.170233  8746 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:22.170707  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.081620 15004 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.111562 14920 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.116848 14943 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.119971 14894 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.117555 14906 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.277614 14712 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.290398 14950 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.294450 14943 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.296720 14920 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.297492  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1048) took 1275 ms. Trace:
I20260709 13:02:22.297890  9440 rpcz_store.cc:276] 0709 13:02:21.021918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.022499 (+   581us) service_pool.cc:224] Handling call
0709 13:02:22.297242 (+1274743us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b540065e854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:22.297437 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54709504}
I20260709 13:02:22.297966 15013 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.193737  8748 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:22.278735  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.300894  9444 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:22.216727  8815 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.302917  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.188058  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.303731  8788 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:22.300151  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.307834  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.308429  8890 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:22.289651  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1072) took 1151 ms. Trace:
W20260709 13:02:22.309995  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.309710  8744 rpcz_store.cc:276] 0709 13:02:21.138083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.138882 (+   799us) service_pool.cc:224] Handling call
0709 13:02:21.155422 (+ 16540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.940862 (+785440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.289323 (+348461us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400465fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:22.289580 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781586816}
I20260709 13:02:22.310699  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.312386  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:22.254933  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.315326  8823 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.317730  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:22.218114  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.318348  8743 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:22.320701  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.321228  8894 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:22.322399  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.304760  8748 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:22.323326  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:22.325740  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.327183  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.327968  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.324210  8749 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.324144  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:22.203166 14961 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.173759 15068 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.277792 15069 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:22.353941  8897 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:22.349618  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:22.351734  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.349238  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.358350  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.358570  9446 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:02:22.359387  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.374114  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:22.241971 15065 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:02:22.326975 14823 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.149s	user 0.010s	sys 0.002s
I20260709 13:02:22.318751  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:22.349823 15066 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:22.377964  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.282984 14904 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.314724 14999 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:22.347294 15033 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:22.288990 14796 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.351157 14943 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:22.384325  8745 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:22.344182 14920 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.347455 15000 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:22.388298  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65058.2us
I20260709 13:02:22.381699  8746 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
W20260709 13:02:22.389976  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.256222 15016 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.399529  8744 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:22.394603  8745 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.375721 14712 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:22.395212  8741 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:22.325491  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:22.376288 14909 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.401503  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:22.422956  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.465124  8745 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.406236  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1053) took 1382 ms. Trace:
I20260709 13:02:22.397423 14920 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.477226 15017 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.468967  9447 rpcz_store.cc:276] 0709 13:02:21.023486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.024272 (+   786us) service_pool.cc:224] Handling call
0709 13:02:21.024537 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.150507 (+125970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.401390 (+1250883us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54006dfbd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:22.406180 (+  4790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288075648}
I20260709 13:02:22.478613 14920 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.480693  8746 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:22.464082 14571 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.482342  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.482195 14571 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.483028  9447 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:22.484449 14571 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.481756  8744 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:22.485908 15070 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:02:22.486583  8744 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:22.491421 14904 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.492125  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:22.491948 14904 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:22.492707 15051 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.003s
I20260709 13:02:22.494122 14904 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.496300 15026 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:22.497246 15071 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:02:22.497624 14899 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.498147 14899 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.500311 14950 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.502126  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1113) took 1374 ms. Trace:
I20260709 13:02:22.502539  9441 rpcz_store.cc:276] 0709 13:02:21.128133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.128269 (+   136us) service_pool.cc:224] Handling call
0709 13:02:22.300682 (+1172413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.300856 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.300879 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.300918 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:22.300944 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.300994 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.501937 (+200943us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540065e850. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:22.502061 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2654114560}
I20260709 13:02:22.387396 15033 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:22.505889 15033 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:22.376149 15068 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.384579 14823 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.510593 14909 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.391232 14999 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.511605  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1082) took 1206 ms. Trace:
I20260709 13:02:22.512629 14909 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.513155 14999 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.357468 15025 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.375298 15061 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.119s
I20260709 13:02:22.585104  9443 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.511973  8747 rpcz_store.cc:276] 0709 13:02:21.305023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.305170 (+   147us) service_pool.cc:224] Handling call
0709 13:02:21.305473 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.775823 (+470350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.511389 (+735566us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400462dd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:22.511549 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839012736}
I20260709 13:02:22.516803  8742 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:22.510927 14823 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.393090 15059 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.598109 15059 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.599563  8818 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:22.600159  9447 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:22.600351 15059 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.525568  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:22.527220  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.543324  9444 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:22.625049  8823 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:22.547073  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.388944 14992 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.589536  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.630918  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.632069  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.632592  8741 raft_consensus.cc:2376] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13. Re-sending same reply.
W20260709 13:02:22.591930 14862 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.001s
I20260709 13:02:22.633600 14862 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.635711 14862 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.523161 14712 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.636776 14712 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:22.637652  9445 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.637929  9449 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.593371 14571 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:22.641538 14571 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:22.641929  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.642005 14571 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.642678 15068 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.643179  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.594743  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2599) took 1445 ms. Trace:
W20260709 13:02:22.472414  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 146975us
I20260709 13:02:22.646776  8890 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:22.600729  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.601547  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.602233  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.602463 14899 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.603189  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.604941 14923 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:22.607618 15016 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.586946  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.607664  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:22.397426 15062 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:02:22.665081  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.612161  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.666190  8818 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.398571 14943 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.615960  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:22.667467  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 51619.1us
I20260709 13:02:22.668279 14943 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:22.669474 14353 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
W20260709 13:02:22.582913 14812 consensus_meta.cc:287] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.802s	user 0.007s	sys 0.003s
W20260709 13:02:22.625097  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.671118 14894 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.670962 14812 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.621992 14909 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:22.672089 14909 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.674083 14909 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.628674  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.674499 15061 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:22.674778  8822 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.675021 15061 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:22.675880 15061 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.620436 15009 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:22.677471  8820 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:22.677664  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.677951 14936 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:22.678992 14936 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:22.679351 14936 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:22.666258  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.680182 14936 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.680811  8821 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:22.681121  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.682391  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:22.682624  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.683418  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.683986  8746 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:22.684383  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.684684  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.684935  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.684383  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.685997  9446 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:22.686501  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.686722  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2315) took 1085 ms. Trace:
I20260709 13:02:22.687151  8821 rpcz_store.cc:276] 0709 13:02:21.600867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.610395 (+  9528us) service_pool.cc:224] Handling call
0709 13:02:21.625894 (+ 15499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.681229 (+1055335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.686509 (+  5280us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400076f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:22.686672 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425649920}
I20260709 13:02:22.687486  8822 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:22.651516  8817 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:22.652083  9441 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:22.689621  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.651537  8816 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.687244  8746 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.690186  8818 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:22.690851  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.691386  8746 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.653559  8891 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.649258  8824 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.658457  8747 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:22.694666  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.518218 15051 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.662201 15033 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.670485  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.678697 14712 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.677500 15009 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:22.673112 14812 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.650106  8889 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:22.654561  8893 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.696607  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.644047  8896 rpcz_store.cc:276] 0709 13:02:21.148892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.149027 (+   135us) service_pool.cc:224] Handling call
0709 13:02:21.149417 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:21.149797 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.149825 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:21.156783 (+  6958us) log.cc:844] Serialized 52 byte log entry
0709 13:02:21.669943 (+513160us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b1800370660. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:21.967900 (+297957us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:21.969055 (+  1155us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.976299 (+  7244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.976447 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.594425 (+617978us) raft_consensus.cc:1685] finished
0709 13:02:22.594525 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.594679 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":204,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":5712,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":57299,"spinlock_wait_cycles":4859264,"thread_start_us":297248,"threads_started":1,"wal-append.queue_time_us":922801}
I20260709 13:02:22.750209  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.697432 14936 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.753413  8743 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.702296 14829 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.702528  8816 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:22.695482  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:22.757359  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61945.4us
I20260709 13:02:22.583657 14904 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:02:22.710493 14353 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:22.709923  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:22.715623  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:22.770968  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61172.6us
W20260709 13:02:22.771080  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 55526.9us
W20260709 13:02:22.719907  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.716308 15043 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.034s	sys 0.062s
I20260709 13:02:22.773511  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.698686  8894 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:22.758606  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:22.774389  8896 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
W20260709 13:02:22.778183  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.740940  8747 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:22.789075  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.746169 14821 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.793373  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.794178  8741 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:22.795859  9444 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:22.795964  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.775215 14829 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.800041 14829 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:22.780694 15043 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.034s	sys 0.062s
I20260709 13:02:22.805507  8896 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:22.746078 14894 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:02:22.795526 15059 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.807730 15059 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.795415 14992 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:22.809652 14992 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:22.810473  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.811111  8742 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:22.812004  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.812633  8742 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.780784 15054 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:22.782126 14796 raft_consensus.cc:2764] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:02:22.782829 15004 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.791525 14812 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:22.819651  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.810491 14992 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.790941 14839 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:22.822844 15059 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.795615 15016 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:22.831614  8741 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:22.835374  8742 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:22.838028  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.848587  8743 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.841944  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.839004 14812 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:22.850698 14812 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.817441 15054 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.852452  8896 raft_consensus.cc:2410] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:22.849550  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.853686  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.856343  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.858086  8743 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.859056  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.860623 14712 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.804371 14821 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.863207  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:22.863068  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 371004us
W20260709 13:02:22.866523  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.846199 15016 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:22.801076 15009 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.870604  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:22.856942 15054 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.867537  8741 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.874290 15016 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.877348  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.877012  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.861636 15034 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:22.881562  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:22.883040  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 1212) took 1568 ms. Trace:
I20260709 13:02:22.883490 14950 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:22.883631  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.884081 14904 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.885514 14862 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:22.893913  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:22.897938  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.898500  8896 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:22.899940  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.901032  8741 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.902606  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.903414  8741 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:22.904484  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.907111  8741 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.888368  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.887477  9445 rpcz_store.cc:276] 0709 13:02:21.314873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.314983 (+   110us) service_pool.cc:224] Handling call
0709 13:02:22.882879 (+1567896us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400499b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:22.882984 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2937357568}
I20260709 13:02:22.890172  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:22.910313 15051 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.912540  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1061) took 1281 ms. Trace:
I20260709 13:02:22.913046  9448 rpcz_store.cc:276] 0709 13:02:21.630702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.630829 (+   127us) service_pool.cc:224] Handling call
0709 13:02:21.635235 (+  4406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.839985 (+204750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.840018 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.840453 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.080407 (+239954us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:22.080495 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.912150 (+831655us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:22.912197 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.912239 (+    42us) raft_consensus.cc:1685] finished
0709 13:02:22.912251 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.912468 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":574,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10251,"spinlock_wait_cycles":345764096,"thread_start_us":232942,"threads_started":1,"wal-append.queue_time_us":321020}
I20260709 13:02:22.917505 15033 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:22.918359 15078 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.166s	user 0.004s	sys 0.000s
I20260709 13:02:22.918681 14950 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:22.919589 14950 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:22.919915 14950 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:22.920253 14950 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.922497  9445 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:22.888959 15009 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.924818  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.925447  8823 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:22.923888  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.931289  8741 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:22.932514  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.933168  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:22.930384  8743 raft_consensus.cc:2376] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18. Re-sending same reply.
I20260709 13:02:22.933965  8816 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.933279  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.934888  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.891810 14909 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:22.937204  8741 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.937397  9448 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19.
I20260709 13:02:22.936172  8743 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:22.930033 14796 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:22.938720 14796 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:22.939733 14796 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.940457 14571 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:22.941279 14796 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.945922  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.946543  8894 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:22.946430  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.947278  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.947371  9449 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:22.947981  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:22.955621  8746 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:22.956557  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.954064  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.955626  8748 raft_consensus.cc:2410] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:02:22.962337 14821 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:22.963061  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:22.961220  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:22.967123 14520 consensus_meta.cc:287] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.873s	user 0.008s	sys 0.004s
I20260709 13:02:22.892128 15034 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.967984 15025 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.968751 14520 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:22.970899 15034 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:22.972506 14520 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:22.973798  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:22.974581  9448 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.973826  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:22.973839  8741 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:22.978139  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.976107  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.893433 14862 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:22.985606  8740 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:22.980330  8743 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:02:22.980485  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.981123  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.984113  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:22.987257  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.987923  8743 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:22.988131  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.988777  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.988718  8894 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:22.989629  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:22.990269  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:22.990830  8894 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:22.855111  8742 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.992220  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.992798  8898 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:22.993950  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:22.994642  8898 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:22.996022  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:22.999508  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.002403  8741 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:23.003212  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.007735  8743 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:23.011734 14796 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:23.012428 14796 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:23.004232  8741 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:23.008379  8816 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:23.015007  8741 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:23.014751  8896 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:23.011200  8742 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
W20260709 13:02:23.025694  8745 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.026s
I20260709 13:02:23.026300 14796 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:23.028380  8745 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.026s
I20260709 13:02:22.937122 14909 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.035357 14909 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.046615  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:23.050132  8894 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:22.962224 14812 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.057381  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:23.059370  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.041842 14899 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.030678 14796 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:02:23.000373 15034 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:02:23.044641 14353 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:02:23.099756  8746 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:23.105417 14906 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:23.108093  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:23.112969  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.115803 15005 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:23.121830  8713 leader_election.cc:400] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19.
I20260709 13:02:23.122781  8741 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:02:23.150050  8741 pending_rounds.cc:85] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:23.150326  8741 pending_rounds.cc:107] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:02:23.150575  8741 raft_consensus.cc:2889] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.130839  8862 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:23.150995  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.149585  8713 leader_election.cc:403] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:02:23.152581  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.157239  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:23.161314  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:23.161957  8894 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:23.157259  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.163743  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:23.164722  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.166527  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.166842  8818 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:02:23.167363  8818 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 7
I20260709 13:02:23.167533  8898 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:23.167689  8818 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:02:23.168188  8818 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.169559  8889 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:23.169605  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.164649  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.172165  8894 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.173615  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:23.165788  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:23.180095  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.182579  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:23.184651  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.186924  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.165661  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:23.190812  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:23.198712  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.201596  8821 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:23.199231  8823 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:23.191471  8748 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:23.204035  8894 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:23.211692  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:23.213732  8824 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:02:23.216979  8748 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:23.150092 14943 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:23.156520 14520 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.153378 14935 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.221241 15031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.032s	sys 0.019s
I20260709 13:02:23.188086 15076 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:23.232456  8822 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:23.257684  8816 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:23.159610 14997 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.270584  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.260540  8746 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:23.198324 14796 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:23.267758  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:23.186614 15059 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.187757 15005 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.274632  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.282446  8824 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:23.275545  8823 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:23.198783 15004 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:02:23.305667  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:23.306628  8822 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:23.318104  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:23.266532 15031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.034s	sys 0.020s
I20260709 13:02:23.324396  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:23.325205  8816 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:02:23.267264  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 377195us
I20260709 13:02:23.273579 14943 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.149523 14862 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.270455 15076 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.358039  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2533) took 1117 ms. Trace:
I20260709 13:02:23.362056  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2532) took 1136 ms. Trace:
I20260709 13:02:23.333830  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:23.334872  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 277574us
I20260709 13:02:23.362644  8749 rpcz_store.cc:276] 0709 13:02:22.225241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.464911 (+239670us) service_pool.cc:224] Handling call
0709 13:02:22.479819 (+ 14908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.480007 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:23.057837 (+577830us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.361596 (+303759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.361662 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:23.361723 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.361844 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.361989 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35950,"prepare.queue_time_us":782133,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":743244,"threads_started":2}
I20260709 13:02:23.365108  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.305738 15004 consensus_queue.cc:1243] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:02:23.270339 14935 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.370680  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:23.375217  8822 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19.
I20260709 13:02:23.358716  8745 rpcz_store.cc:276] 0709 13:02:22.240455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.469276 (+228821us) service_pool.cc:224] Handling call
0709 13:02:22.469638 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.469927 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:23.031166 (+561239us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.357624 (+326458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.357678 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:23.357719 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.357786 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.357985 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4250,"prepare.queue_time_us":740881,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":1821,"raft.queue_time_us":2730,"thread_start_us":842348,"threads_started":2}
I20260709 13:02:23.341094  8816 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:23.373571 15054 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:23.387259  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.323460 14353 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:23.269271 14520 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.394740  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:23.355523 14943 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:23.424129  9448 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:23.449015 14954 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:23.359642 15076 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:23.368299 15026 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:23.467319 15090 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.367188 14571 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:23.375653 14712 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.323356 15005 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:23.524363 14904 consensus_meta.cc:287] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.002s
I20260709 13:02:23.524904 14954 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:23.537636 14812 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:23.543875 14520 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:23.548134 14909 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.000s
I20260709 13:02:23.461921 14353 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.550911 14935 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:23.562974 15054 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:23.479532 15016 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.588s	user 0.003s	sys 0.005s
W20260709 13:02:23.576229  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.579262  8823 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:23.560838 14796 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.584658 15005 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:23.585805 14812 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:23.599339  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.588471 14904 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.570791 15026 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:23.589798 14571 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:23.616532 14615 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:23.633445 14615 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:23.633713  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 422108us
I20260709 13:02:23.611107 14935 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:23.640059  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 389750us
I20260709 13:02:23.642221  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:23.637930  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:23.644301  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:23.633927 14615 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.654778  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:23.607328 14909 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:23.674901  8745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:23.678097  8745 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.675076  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:23.616834 15016 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.666955 15054 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:23.671908 14996 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:23.708578 14950 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.788s	user 0.010s	sys 0.003s
I20260709 13:02:23.740135  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1091) took 1224 ms. Trace:
I20260709 13:02:23.682679  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:23.709410 14353 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:23.720285 14992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.007s	sys 0.023s
I20260709 13:02:23.670253 15031 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.720508  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45522.1us
I20260709 13:02:23.782364  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.666775 14520 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.712060 15009 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.745443  8742 rpcz_store.cc:276] 0709 13:02:22.516197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.021541 (+505344us) service_pool.cc:224] Handling call
0709 13:02:23.021817 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.022029 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:23.323780 (+301751us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.739641 (+415861us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.739723 (+    82us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:23.739789 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.739888 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.740056 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":280944,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":481,"thread_start_us":716376,"threads_started":2}
W20260709 13:02:23.787392  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.787650  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.789178  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:23.791953  8744 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.015s	sys 0.035s
W20260709 13:02:23.792335  8744 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.015s	sys 0.035s
I20260709 13:02:23.744134 15054 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:23.767940  9445 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:23.777624 14992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.008s	sys 0.023s
I20260709 13:02:23.721539 15004 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.714427 15021 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:02:23.716442 15026 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:23.794060  8863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.014s	sys 0.038s
W20260709 13:02:23.748878  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.785894 15009 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.784219 14520 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.765839 14950 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:23.796932  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 463045us
I20260709 13:02:23.810891  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:23.794014 14894 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:23.794193  8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.011s	sys 0.025s
I20260709 13:02:23.813992  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:23.814247 14943 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:23.814728 15096 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:02:23.798768 14961 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:23.815429 15100 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:02:23.809324 15072 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.013s	sys 0.033s
I20260709 13:02:23.816205 15105 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:23.812165  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:23.820729  8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.012s	sys 0.025s
I20260709 13:02:23.819032  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:23.834610  9440 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:02:23.822970 15026 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:23.844040  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.846359 14812 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.846853 14812 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.848953 14812 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:23.851207  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.831807 15072 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.013s	sys 0.033s
I20260709 13:02:23.856158 15059 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:23.856683 15059 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:23.857733 14812 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.857679 15059 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.858363 14812 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.823087  9444 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:23.824981  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:23.862212  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.862627  9442 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:23.811964  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 417090us
I20260709 13:02:23.803638 15005 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:23.815279 14992 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.816066  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1089) took 1421 ms. Trace:
I20260709 13:02:23.816962 15009 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:23.892683  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:23.810472 15104 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.819073 14520 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:23.833169  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:23.902009  9441 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:23.835661  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:23.890266  8743 rpcz_store.cc:276] 0709 13:02:22.394765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.012558 (+617793us) service_pool.cc:224] Handling call
0709 13:02:23.012860 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.013064 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:23.359754 (+346690us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.806766 (+447012us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.815740 (+  8974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.815803 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:23.815852 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.815918 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.816015 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14348,"prepare.queue_time_us":275303,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":52328,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":12228,"spinlock_wait_cycles":15146752,"thread_start_us":786658,"threads_started":2}
W20260709 13:02:23.903941  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 68295.8us
I20260709 13:02:23.794126 15031 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.836714 15105 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.837534 15090 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:23.837846 14712 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:23.918282 14712 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:23.840723 15080 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.738s	user 0.006s	sys 0.007s
I20260709 13:02:23.922711  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2317) took 2321 ms. Trace:
I20260709 13:02:23.923137  8819 rpcz_store.cc:276] 0709 13:02:21.601491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.610594 (+  9103us) service_pool.cc:224] Handling call
0709 13:02:21.610889 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.611243 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.611270 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.611681 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.598391 (+986710us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b1800482380. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:22.836384 (+237993us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:22.836425 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.836492 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.840099 (+1003607us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:23.922439 (+ 82340us) raft_consensus.cc:1685] finished
0709 13:02:23.922466 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.922615 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4782,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":11834,"spinlock_wait_cycles":534403584,"thread_start_us":237813,"threads_started":1,"wal-append.queue_time_us":1459862}
I20260709 13:02:23.819332 14906 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:23.925627 14906 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.926900  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2256) took 1260 ms. Trace:
I20260709 13:02:23.927279  8747 rpcz_store.cc:276] 0709 13:02:22.666849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.795694 (+128845us) service_pool.cc:224] Handling call
0709 13:02:23.926659 (+1130965us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000b5754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:23.926809 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316172032}
W20260709 13:02:23.841924  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.859006 15059 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:23.793654  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1086) took 1480 ms. Trace:
I20260709 13:02:23.928764  8744 rpcz_store.cc:276] 0709 13:02:22.313563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.487312 (+173749us) service_pool.cc:224] Handling call
0709 13:02:22.487603 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.487798 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:23.074242 (+586444us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.793215 (+718973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.793269 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:23.793398 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.793473 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.793596 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129118,"prepare.queue_time_us":791155,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":24846,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":4242,"thread_start_us":1148941,"threads_started":2,"wal-append.queue_time_us":830143}
I20260709 13:02:23.864604  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:23.865898  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:23.819527 14894 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:23.933452  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:23.833734  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 697070us
W20260709 13:02:23.857834 15097 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:02:23.809245  8863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.014s	sys 0.039s
I20260709 13:02:23.834645 14961 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:23.905898  8896 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:23.906469 15111 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:02:23.907426 14904 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:02:23.894699 14997 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 13:02:23.861450 14812 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:23.904978 14571 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:02:23.915904 14935 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:23.917654 14950 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:23.930902  9449 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
W20260709 13:02:23.930933  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66416.7us
I20260709 13:02:23.945503  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:23.894881 15005 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.948061  8894 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:23.951242  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:23.951521  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.951922  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.952310  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.931550 14970 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:23.931967  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:23.936486  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.898119 14909 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:23.940331  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 749598us
I20260709 13:02:23.942999 14997 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19.
I20260709 13:02:23.957149  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:23.958657  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.939201 15090 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:23.962471  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:23.963318  8893 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:23.898339 15054 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:23.972672  8898 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:23.970235  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:23.970253  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.974267  8889 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:23.975114  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 186037us
I20260709 13:02:23.955948 15068 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:23.953714 14970 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.955713 15103 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:23.957127 14815 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:23.982371  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 299764us
I20260709 13:02:23.934464 15105 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.007298  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.007573  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:23.989085  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1088) took 1675 ms. Trace:
I20260709 13:02:24.008481  8748 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:24.008394  8740 rpcz_store.cc:276] 0709 13:02:22.314143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.000410 (+686267us) service_pool.cc:224] Handling call
0709 13:02:23.043436 (+ 43026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.043772 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.043802 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.044214 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.368844 (+324630us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:23.368896 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.368983 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.988873 (+619890us) raft_consensus.cc:1685] finished
0709 13:02:23.988894 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.989031 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5840,"thread_start_us":313990,"threads_started":1,"wal-append.queue_time_us":725626}
I20260709 13:02:23.962308 14950 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:23.969671 14935 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.980165  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:23.993875  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:23.994149  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.013216  8747 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:23.996538  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.998894  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.013576  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.013919  8745 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:24.000276  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:23.991127  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:23.971655 14909 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:23.995267  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.015434  8891 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:24.017257  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:24.017999  8893 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:24.018026  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.021476  8740 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:23.975152 14996 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:24.023715  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.040198  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.031400  8747 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:23.976105 15116 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.041821  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.042688  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.044013  8746 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:23.990370 14823 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.990397 14970 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:23.978389 15068 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:24.049325  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 69220.8us
I20260709 13:02:24.050504  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2542) took 1578 ms. Trace:
I20260709 13:02:24.057842  8741 rpcz_store.cc:276] 0709 13:02:22.471659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.015808 (+544149us) service_pool.cc:224] Handling call
0709 13:02:23.122330 (+106522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.151031 (+ 28701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.050310 (+899279us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540016e9d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:24.050448 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244218624}
I20260709 13:02:24.058133  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:24.058947  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.014513 14950 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.068818  8890 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:24.069375  8891 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:24.011670 14992 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:24.072139  8893 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:24.070381  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:23.989128 14997 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:02:24.077028  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:23.962884 14894 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.083135 14992 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.083824 14894 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:24.046402 15004 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:24.038668 14839 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.085680 14992 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.072122 15090 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.091431  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.098515  8897 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:24.129590  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.011960 15104 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:02:24.138325  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.140050  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.140650  8897 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:24.073050 14950 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:24.039755 15021 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:24.011785 15103 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.153869  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.154943  8816 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:24.134651  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.156992  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.137463  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.159471  8823 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:24.137732  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.160389  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:24.022920 14935 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.075824 14909 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:24.199177  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.158653  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:24.053622 15009 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.087888 15118 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:24.213457  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 278052us
I20260709 13:02:24.094852 15051 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:24.133078 15111 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:24.067206 14823 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.225116  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.229514  8816 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:24.207121  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 273779us
I20260709 13:02:24.129594 15004 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:23.990463 14961 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.216161  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.132709 14839 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:24.206554 15077 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.073s	user 0.001s	sys 0.012s
W20260709 13:02:24.206532 15028 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.008s	sys 0.027s
W20260709 13:02:24.152257 15109 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 13:02:24.238979  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 229900us
I20260709 13:02:24.147712 15025 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.286252 15025 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.286617  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.058282 15116 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.242460  8744 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
W20260709 13:02:24.264245 15088 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:02:24.262831  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 221128us
I20260709 13:02:24.301034  8815 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:24.275764  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.205605 15124 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:24.305817  8815 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:24.291983  8741 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:24.281337 15028 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.008s	sys 0.027s
I20260709 13:02:24.211596  8819 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:24.302335  8817 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:24.238529 15004 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.263950 15123 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:24.262475 14829 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.311491  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:24.293123 15110 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.350s	user 0.001s	sys 0.000s
I20260709 13:02:24.318899  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.319900  8819 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:24.321161  8816 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:24.321391  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.324779  8823 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.316762  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:24.301910 15092 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:02:24.326933  8715 leader_election.cc:400] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19.
I20260709 13:02:24.337954  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.338158  8741 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:24.338814  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.304111 15118 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:24.332860  8823 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:24.209048 14996 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.339587  8740 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:24.341280  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.384366  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.341804  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.386009  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.386591  8745 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:24.387393  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.388139  8745 raft_consensus.cc:2410] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:24.388780  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.328330  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.302769 14571 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.299762 14904 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:24.345732  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.347337  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.344501 15133 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.206866 15068 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.390086  8749 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:24.344673  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:24.389961  8824 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:24.339599 15119 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:02:24.396736  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.342365 15005 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:24.351986  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 193442us
I20260709 13:02:24.352104  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.316938 15123 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:24.315088 15004 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.318238 14917 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:24.349947 14353 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:02:24.355674 15127 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:24.357854 15025 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.358616 15026 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:24.362830  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.334782 14997 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:24.336259 14650 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:24.318146 15131 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:24.377345  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 93072.8us
I20260709 13:02:24.341555  8748 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:24.390259 14904 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.279351 14839 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.391045  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:24.390352 15133 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.312268 15104 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:24.400552  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.397964  8746 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
W20260709 13:02:24.366278 15076 consensus_meta.cc:287] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.790s	user 0.003s	sys 0.009s
I20260709 13:02:24.346354 15139 raft_consensus.cc:2764] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:24.341087 15132 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.323340 14829 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:24.465581  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 140067us
I20260709 13:02:24.453827 15131 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.433084  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.340211 15122 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:02:24.435675  8745 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:24.396406 15105 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.438993  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:24.536296  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 135825us
W20260709 13:02:24.511111 14615 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.793s	user 0.007s	sys 0.003s
I20260709 13:02:24.537765  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:24.441443  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.429572 14909 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:02:24.541666 15140 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:02:24.541996 15028 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:24.542400 15028 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:24.542963 14909 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:24.543268 15028 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.527918 15123 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.544317 15028 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:24.442369  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 98275.3us
I20260709 13:02:24.547451  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:24.547775  8743 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
W20260709 13:02:24.455881  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.515s (SENT) [suppressed 35 similar messages]
W20260709 13:02:24.548766  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 92973.2us
I20260709 13:02:24.430567 14917 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:24.471208  8749 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:24.498621 14823 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.469856 14935 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.455631 15026 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.540457 15103 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:24.540918 15128 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:24.550347  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.551016  8745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:24.532712 15131 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.455819 15025 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:02:24.576836  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.578174  8745 raft_consensus.cc:606] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.543395 14909 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.456681 14650 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.585328  8745 raft_consensus.cc:614] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:24.574018 15005 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.533828 15009 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.531412 15144 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:24.589964 15025 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:24.608439  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:24.598760  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48486.4us
I20260709 13:02:24.599445  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.600560 15116 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.601691 14906 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.593959 14909 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.520006 15076 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.605671 15021 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:24.642840 15021 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.643939  8894 raft_consensus.cc:2368] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22.
I20260709 13:02:24.616645 14943 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.607107 15133 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.607744 14862 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:24.609592 14829 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:24.614316 14712 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.000s
I20260709 13:02:24.646431 14712 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.648705 14712 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:24.605255 15059 consensus_meta.cc:287] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.001s
I20260709 13:02:24.627472  8749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:24.649662  8749 raft_consensus.cc:606] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.650007  8749 raft_consensus.cc:614] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:24.610188 15137 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:02:24.594224 15105 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:24.651078 15105 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.615388 14839 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.621986 15149 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:24.622671  8822 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:24.531957 14353 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:02:24.625144  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.627090 15001 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:02:24.592528 15143 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.643384 15076 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.605544 15157 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.633025 14923 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:24.608148 15132 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.610539 14917 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.653769  8742 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:02:24.653245  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.676266 15021 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.706445 15059 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:24.707114 15059 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.707846 14353 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.708379 14353 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.710494 14353 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.622463 15025 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:24.711637 15025 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:24.558756  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 119987us
I20260709 13:02:24.674197  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.656351  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:24.509228 14970 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.677140  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.678993  8818 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:02:24.680951  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:24.680997  8816 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:02:24.689486 14894 consensus_meta.cc:287] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.602s	user 0.006s	sys 0.006s
W20260709 13:02:24.687563  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62515.2us
I20260709 13:02:24.721611  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.632392 15009 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.614845 14650 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.692723 15143 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.553222 14615 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.616195 15026 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:24.696619 15105 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.697031 15154 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:24.682837 14829 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:02:24.714694  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:24.714841  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58599.1us
I20260709 13:02:24.684767 15118 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:24.726323  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.738672  8822 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:24.632611 15144 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.712007 14923 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:24.732826  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:24.740872 14650 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:24.741621  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.762185  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.762486  8818 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:24.722142 14353 raft_consensus.cc:2764] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:24.742492  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.764356  9412 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:24.743003 15026 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:24.765735  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.767403  8816 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:24.693023 15161 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.715741 15031 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:24.728137 14904 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.765188  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.720384 15059 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.734354 15143 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.766065  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:24.777056  8823 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:24.763826 14650 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.743995  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.732393 15154 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:24.767284 14823 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:02:24.773038 15098 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.727497 15021 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.747289 15025 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:24.762296 14923 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:24.779486  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.780506  9445 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:24.780695  9448 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:02:24.793566  9448 pending_rounds.cc:85] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Aborting all ops after (but not including) 9
I20260709 13:02:24.793972  9448 pending_rounds.cc:107] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:24.794318  9448 raft_consensus.cc:2889] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.768685  8820 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:24.778682 15156 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:24.781003  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:24.765710 15133 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:24.783318  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.731573 15151 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:24.786041  9444 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:24.732877 15076 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:24.767660  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:24.794978  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:24.813231  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:24.787572 15111 consensus_meta.cc:287] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.005s
I20260709 13:02:24.712430 15157 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.769647 14862 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:24.730212 15009 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.797331  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.799839  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:24.806252  8823 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:24.761965 14943 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:24.892088  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:24.808550  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 216908us
I20260709 13:02:24.769007 15147 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.810264  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:24.888891  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 118782us
I20260709 13:02:24.810658  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:24.812062  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:24.893980  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.812669  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:24.790478  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.894590  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.894826  8893 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:24.893103  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:24.895177  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.895795  8892 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:24.895897  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.896744  8893 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:24.791342  8890 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:24.769544 15159 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:24.774245 15150 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:24.899587  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 99866.1us
I20260709 13:02:24.902370  8891 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:02:24.793058 14520 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.882s	user 0.012s	sys 0.000s
I20260709 13:02:24.787354 14650 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:24.782605 15142 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.792135 14906 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.106s
I20260709 13:02:24.888378  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:24.897964  9449 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
W20260709 13:02:24.874467  8861 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.7.117.190:45111, user_credentials={real_user=slave}} blocked reactor thread for 65250.6us
I20260709 13:02:24.778683 14829 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
W20260709 13:02:24.884414  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101212us
I20260709 13:02:24.804396 14904 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 13:02:24.887971 15111 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:24.915206  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:24.792092 15136 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 13:02:24.914625  8895 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:24.896908  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:24.873422 15105 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:02:24.906890 14862 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:24.885103 14712 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:02:24.883121 14961 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:02:24.811527 15098 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.911163 14520 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.963343 14909 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:24.985857 15144 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.909912 15152 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:24.908550 15158 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:24.993983 15166 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.003463 15166 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.005694 15166 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:24.920650 15150 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:25.007408  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.007853 15150 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:25.000826  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 85698.9us
I20260709 13:02:25.010607 15158 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.011042 15158 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.011134  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:25.014540  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 246949us
I20260709 13:02:24.969565 15031 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.017515  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:24.971846 15160 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:25.026742 14353 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.882736 15118 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:24.994136 14823 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:25.039680  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 145154us
I20260709 13:02:25.062461  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.063627  8789 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.033141  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:25.044184  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.046149 15157 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.047415 15009 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.047415 14353 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.048808 14862 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.023149 15016 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.031090 15168 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.054272 14615 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:25.126945  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 115890us
I20260709 13:02:25.030313 15159 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.127700  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.055527 14520 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.055696  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:24.973044 15154 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:25.056205 15028 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.501s	user 0.001s	sys 0.008s
I20260709 13:02:25.034224 15031 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.065124  8789 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.129715  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:25.065868 15166 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:25.097342 15004 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 13:02:25.142795 14909 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.065018 15131 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.143322 14650 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:25.145498  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80441.6us
I20260709 13:02:25.153503  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:25.153841  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.160967  8789 leader_election.cc:400] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:25.161446  8789 leader_election.cc:403] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:25.155416  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.162462  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.162653  8747 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.154462  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:25.155933  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.078938 14823 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:25.155709  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:25.125487 15172 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.158007  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.144656 15149 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:25.169261  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.170259  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:25.170912  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.163861  8740 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.160871  8742 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.065454 15173 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:02:25.133419 15170 raft_consensus.cc:2764] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:25.148382 14615 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:25.176608  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.178310  9443 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:25.179261  8895 raft_consensus.cc:2376] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17. Re-sending same reply.
I20260709 13:02:25.219269  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.219998  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:25.167034 15154 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.169178 15076 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:25.220829  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.223196  8895 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 12.
I20260709 13:02:25.164434 15160 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.224061  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.220319 14894 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:25.225167  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.225771  8895 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:25.226562  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:25.227161 15165 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:02:25.227134  8895 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:25.174602 15166 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:25.230248  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.230818  8895 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:25.235463 14923 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.238229 14650 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.233268  8895 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:25.239626  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.159694 15004 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.240934  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 1086) took 1290 ms. Trace:
I20260709 13:02:25.241348  8894 rpcz_store.cc:276] 0709 13:02:23.950873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.950990 (+   117us) service_pool.cc:224] Handling call
0709 13:02:25.240757 (+1289767us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b5400248954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:25.240881 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591371136}
W20260709 13:02:25.238448 15146 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.064s	user 0.005s	sys 0.013s
I20260709 13:02:25.149044 15152 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:02:25.244064  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:25.229733 14950 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.002s
I20260709 13:02:25.237128 15138 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:25.297989 15138 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:25.298979 15138 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.300315 15138 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.302557 14923 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.251206 14353 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.182503 15143 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.305168 14823 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:25.305646 14823 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:25.305711 15143 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.306542 14823 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.307197 14950 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.309782 14997 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:25.310278 14997 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:25.311854 15133 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:25.312368 15133 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.311928 14997 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.252339 15160 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.247532 15076 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.316392 15076 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.255610 15169 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.134932 15168 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.158571 15158 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.256975 15175 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:02:25.259753 15115 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:25.261927 15177 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:25.243391 15159 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.266316 15123 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:25.279521 15149 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.280913 15131 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.249707 15004 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.330173 15115 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.309103 14970 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.331565 14997 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:25.313145 15133 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.338037 15134 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:25.347067 15133 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.344084 15134 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.350347 15168 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.351346 15134 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.351761 15158 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.326572 15169 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.354228 15169 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.392050  8744 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
W20260709 13:02:25.394860  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.408248  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 245687us
W20260709 13:02:25.410605  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.412810  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.415558  8817 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:25.416329  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:25.416260  8787 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:25.417551  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:25.420238  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 254941us
I20260709 13:02:25.420611  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.422889  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.423645  8889 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:25.423952  8894 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:25.425308  8891 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:25.420782  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.413228  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.428306  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.429582  8816 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:25.430475  8819 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:25.433204  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.436049  8789 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.433743  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:25.443717  8816 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:25.447490  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:25.449687  8816 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.450686  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:25.326262 14823 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.453881 14353 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.454965 14353 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.300213 14571 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:25.511145 14353 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.512259  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:25.513334  8819 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.516634 15142 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.517452 15142 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.520526 15142 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:25.521118  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.526278  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90458.9us
I20260709 13:02:25.452925  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:25.497038  8897 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:25.509613 15115 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.533999  8894 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:25.540295 15168 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:25.541254  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.542047 15173 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:25.543157 15173 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:25.543347  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:25.541172 15168 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:25.543741 15173 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.522884  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:25.554105  8819 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.564703 14812 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:25.566067  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 113393us
I20260709 13:02:25.566690 14923 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:25.567198  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:25.569473  8740 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.571807 15145 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.572412 15145 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.572189  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.572945 15145 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.573479  9449 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.574816  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:25.569561  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43607.7us
I20260709 13:02:25.577247  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.578751  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1080) took 1287 ms. Trace:
I20260709 13:02:25.578962  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:25.579416 15115 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.579313  8820 rpcz_store.cc:276] 0709 13:02:24.290673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.290803 (+   130us) service_pool.cc:224] Handling call
0709 13:02:24.291163 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.795919 (+504756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.578090 (+782171us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:25.578503 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097819392}
I20260709 13:02:25.580466  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:25.581743  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77688.9us
I20260709 13:02:25.583252  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.586944  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:25.589634  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.594210 15157 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.594065  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:25.598058  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.599613 15159 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:25.606416  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:25.607036 15145 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.608918  8824 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:25.609134  9442 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:25.611968 15123 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.522944 14950 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.551131 15168 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:25.621225  8816 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:25.553385 15173 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:25.552592 14571 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.523299 15158 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:25.640035  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.641554  9449 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:25.647094  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46851.3us
I20260709 13:02:25.635087 15159 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.567235 14923 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:25.650750 15157 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.652125  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:25.655500  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:25.663926  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 85094.3us
I20260709 13:02:25.664562 15183 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:25.665355  9448 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.665747  9445 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.666751 15133 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.667658  9441 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:02:25.668962 15076 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:25.670009  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56728.4us
W20260709 13:02:25.670599  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.640014 14812 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:25.682173  9440 raft_consensus.cc:2393] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:25.682677  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 139622us
I20260709 13:02:25.676573  9444 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:25.675470 15159 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:25.688385 15183 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.700202 15183 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:25.701669 15076 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.705380  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:25.677021 14923 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:25.715092  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:25.716039  8818 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:25.717501 15051 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.718436 15051 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.719529  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.720523  8816 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:25.722221  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:25.723690  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62712.2us
I20260709 13:02:25.723362 15133 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.717562 14839 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.724534  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.724900 14839 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.725755 15160 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:25.726066 15133 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.726193 14796 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:25.726709 14796 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:25.727247 14796 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:25.728045 14894 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.008s
I20260709 13:02:25.728618 14894 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:25.689554 14812 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:25.729346  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60587.1us
I20260709 13:02:25.729475 14812 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:25.730216  8742 consensus_meta.cc:287] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.009s
I20260709 13:02:25.730705  8742 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:25.731165  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:25.731693  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.732373 15169 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:25.722461  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:25.733072 15169 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.733893  8740 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:25.734740 15145 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.723412  8746 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:25.735740 15169 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.736475 15169 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:25.737182 14917 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.724021  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:25.737788 15123 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:25.749756 15123 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.752274 15123 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.738765  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:25.738935  8745 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:25.741801 15183 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:02:25.737334 15169 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:25.733757 14839 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.737655  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.765550  8822 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 14: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:25.765657  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.704887 15076 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.785578  8819 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:25.783027  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.761438 15169 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.803432  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.696795 15142 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.101s
I20260709 13:02:25.748548 14812 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.828233 14571 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.834038  9448 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.835208  8824 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:25.838389  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:25.847519  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 106442us
I20260709 13:02:25.849773  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.850749  8746 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
W20260709 13:02:25.851320  8889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 1093) took 1844 ms (client timeout 2208 ms). Trace:
W20260709 13:02:25.851750  8889 rpcz_store.cc:269] 0709 13:02:24.006791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.007305 (+   514us) service_pool.cc:224] Handling call
0709 13:02:25.851079 (+1843774us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540046da54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:25.851258 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3253147520}
W20260709 13:02:25.856365 15192 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 13:02:25.857893 15098 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.852128  8896 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:25.849864  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:25.845464  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.897037  8893 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:25.853008  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:25.902591  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.905934  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:25.907989  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.908993  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.915066  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:25.917011 15166 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:25.917179  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.917722 15166 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.918655  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:25.919878 15166 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.920084  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.921263 15108 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:25.845464 15123 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.922215 15108 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:25.930938  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101358us
I20260709 13:02:25.932751 15108 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.929031  8745 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:25.933708  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.912287  9448 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:25.834643 14894 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:02:25.933995  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:25.934499  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 195861us
W20260709 13:02:25.936894  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 143077us
I20260709 13:02:25.935457  8824 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:25.934963  9441 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:25.944165  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:25.958763  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:25.933264 15138 consensus_meta.cc:287] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.011s
I20260709 13:02:25.936651 14839 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.974957 15157 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:25.975936 15145 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:25.976091 15134 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:25.978612  8745 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:25.979744  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.968021  8787 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:25.933686 15123 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:25.969018  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.981925 14894 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:25.982080  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.982548 14894 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:25.969414  8741 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:25.983335 15123 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:25.984215  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.983701 14894 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:25.969900  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.970379 14571 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:25.971371 15190 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.003s
I20260709 13:02:25.982758  8891 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 13.
I20260709 13:02:25.982710  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:25.989872  8893 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:25.995633  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:25.997169  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.997449 14615 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:25.962235 15108 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.001895  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:25.958185 14992 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:26.050211  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.050015  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.054551  8746 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:26.054122  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:25.998701 14812 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.059015  8745 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:26.066891  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71381.1us
I20260709 13:02:26.032037 15157 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.028363 15076 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.073125  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.030639 15108 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.077230  8890 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.082934  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:26.091670  9445 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:26.096863  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:26.104785  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.096074 15108 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.114358  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.078151 14899 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:26.118686  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 224141us
I20260709 13:02:26.120280  8889 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:26.116333  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71700.5us
I20260709 13:02:26.115883  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:26.124366  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.129639  8891 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:26.117718 15115 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:26.137403  8894 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:26.141487 15201 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:26.100418 15157 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.154290  8894 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:26.108949 15183 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:02:26.160655  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:26.147037 15115 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:26.111905 14796 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.150660 15134 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:26.170553  8740 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:26.150460 15166 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:26.197126  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.196879 15166 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.199610 15166 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.200498 14823 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.201261 14823 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.203928 14823 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.204424 14909 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.227730 15159 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:26.169173 14997 consensus_meta.cc:287] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.010s
I20260709 13:02:26.173220 15004 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:02:26.171949 15173 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.174837  8893 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:26.128937 15076 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:02:26.175565  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 325895us
I20260709 13:02:26.177414  8898 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:26.178231  8747 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:26.151764 15133 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:02:26.171103 15115 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.183802  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:26.186805 14796 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.173521 14821 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:26.194060  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1076) took 1955 ms. Trace:
I20260709 13:02:26.196192 15157 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.229890 14839 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.233103 15134 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:02:26.186870  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 180723us
I20260709 13:02:26.163247  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.125557 14923 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.264932  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.271387  8892 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:26.271791  8749 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:26.272866 14839 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.274080 15173 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.275216 14839 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.268816 14821 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.277002  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:26.277585  8742 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:26.278447  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:26.280187  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.155522 15169 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.248044  8889 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:02:26.236809  8789 connection.cc:256] RPC call timeout handler was delayed by -1.9548150890000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
I20260709 13:02:26.262193  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.265859  8745 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:26.283171  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.287712  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.285313  8745 pending_rounds.cc:85] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:26.286861 15169 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:26.288515  8745 pending_rounds.cc:107] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:26.297536  8745 raft_consensus.cc:2889] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.301779  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.302313  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.303592  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.292773 15157 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:26.293246 14997 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:26.308873 14862 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.309731  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.309911  8893 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:02:26.310320 15197 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:02:26.308967  8816 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
W20260709 13:02:26.310348  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 140753us
I20260709 13:02:26.303848 15145 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.318637 15145 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.320803 15145 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.300315 15169 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.310521 14923 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.293246 14839 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.255975  8821 rpcz_store.cc:276] 0709 13:02:24.238626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.239165 (+   539us) service_pool.cc:224] Handling call
0709 13:02:24.279821 (+ 40656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.280329 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:24.302043 (+ 21714us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.282223 (+980180us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b1800442060. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:25.584496 (+302273us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b1800442060. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:02:25.906609 (+322113us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.915913 (+  9304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.916608 (+   695us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.916735 (+   127us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:25.916771 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.916869 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.193641 (+276772us) raft_consensus.cc:1685] finished
0709 13:02:26.193664 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.193873 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132661,"prepare.queue_time_us":37494,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":116916,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":13288,"spinlock_wait_cycles":27950208,"thread_start_us":178786,"threads_started":1,"wal-append.queue_time_us":1821029}
I20260709 13:02:26.296093 15004 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:26.301429 15117 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.302774 15076 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:26.337332  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.329996  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.340276  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.342181 15133 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.342744 15133 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:26.345031 15203 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:26.345072 15133 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.354527  8712 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:26.312590 15157 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.341274  8822 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:26.353641  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.359557 14862 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.362469 14862 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.313486 14997 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.330502 14839 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.368024  8889 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:02:26.354324 15168 consensus_meta.cc:287] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 13:02:26.371477  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:26.371977  8824 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:26.339195 15076 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:26.372593  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.374770  8822 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.375362  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:26.365476 15157 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.383675  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:26.367827 14839 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.366997 14997 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.386376  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.387622  9445 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:26.391409  9444 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:26.391490 15206 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.392092  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.392793  8745 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.389055  8749 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:26.394874 14796 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:26.395494 14796 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.376740 15076 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:26.399051 15205 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:26.402221  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.404538  8824 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:26.405108 15207 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.415145  8745 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:26.392056 15206 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.424746  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:26.425336  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.515s (SENT)
W20260709 13:02:26.425783 15209 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:02:26.426102  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:26.427407 14862 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:02:26.429872  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112592us
I20260709 13:02:26.429970  8745 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:26.430652  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.374202 15168 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.432627  8824 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:26.434233  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:26.435966  8821 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:26.439903  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.443106  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.457008  8822 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:26.458734  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.372056 15117 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.458890  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.418247 15206 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.464560  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.465571  8745 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:26.429522 15115 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.470690  8816 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:26.473129  9446 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:26.446604 14796 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.475613  8820 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:26.449828 15133 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:02:26.476336 15207 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:26.477681  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.466446 15004 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.482542 15169 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.482481 15004 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.483659  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.484685 15207 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.485172 15207 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.485597 15207 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.473697 14862 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:26.478744 15115 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.479666  8894 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.480086  8749 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:26.485419  8817 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:26.449690 15108 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:26.517446  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43892.5us
I20260709 13:02:26.518368  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:26.519464 14997 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.520067  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:26.521051  8788 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:26.484740 15169 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.466457 14571 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.522315 14571 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.523518 15117 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.524295 15117 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:26.524662 15117 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.525010 15117 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:26.485188 15004 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.527158  8822 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:02:26.626585  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 269720us
I20260709 13:02:26.526376  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.736047  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.741448  8749 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:26.735750  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.748206  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:26.753623  8821 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:26.525484 15159 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.757835  8820 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:02:26.509510 14862 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.597388 15212 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:26.635026 15115 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.589773 14997 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.750536 15169 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.605793 15147 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:26.743242 15108 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.525810 15111 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.601544 14923 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.778730  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:26.750519 14615 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.784161  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:26.485423 14992 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.288s
I20260709 13:02:26.595996 15207 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.772830 14862 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.802745  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:26.804980  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68931.2us
W20260709 13:02:26.809316  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 323083us
I20260709 13:02:26.810050  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:26.758553 15157 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.785424 14796 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:26.817903  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 334314us
I20260709 13:02:26.595788 14839 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.772462 15004 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:26.820868 14571 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.828866 15147 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:26.844113 15159 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:26.849068 15159 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.851575 15159 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.854878  8745 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:26.855824  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.834673 15212 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:26.860780  8824 raft_consensus.cc:2393] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:26.810832 15115 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.804349 15169 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.816802 14923 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.873191 15098 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:26.873807 15098 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:26.875003 15098 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.875860 15169 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.876405 15098 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:26.878362  8743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2551) took 2440 ms (client timeout 1644 ms). Trace:
W20260709 13:02:26.878923  8743 rpcz_store.cc:269] 0709 13:02:24.338080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.345072 (+  6992us) service_pool.cc:224] Handling call
0709 13:02:26.758325 (+2413253us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540016e4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:26.778129 (+ 19804us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362638080}
I20260709 13:02:26.879725  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1118) took 1139 ms. Trace:
I20260709 13:02:26.880266  8894 rpcz_store.cc:276] 0709 13:02:25.740816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.158540 (+417724us) service_pool.cc:224] Handling call
0709 13:02:26.166689 (+  8149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.511054 (+344365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.879370 (+368316us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54000aa5d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:26.879632 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719279872}
I20260709 13:02:26.884799 15031 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.886643  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 1234) took 1306 ms. Trace:
I20260709 13:02:26.894222 15218 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:26.897190  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1090) took 1309 ms. Trace:
I20260709 13:02:26.898883 15138 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.899314 15217 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:26.899626 15004 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:26.900190  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65616.3us
W20260709 13:02:26.900522  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.929790  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:26.926581  9446 rpcz_store.cc:276] 0709 13:02:25.579841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.582722 (+  2881us) service_pool.cc:224] Handling call
0709 13:02:26.886344 (+1303622us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b5400559954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:26.886532 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960115584}
I20260709 13:02:26.931317  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:26.932646 15173 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:26.900265 14992 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:26.933259 15173 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.934132 14992 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:26.934514 14992 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:26.934864 14992 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:26.935395 15173 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.907459  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2937) took 1301 ms. Trace:
I20260709 13:02:26.937444  8819 rpcz_store.cc:276] 0709 13:02:25.606292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.606989 (+   697us) service_pool.cc:224] Handling call
0709 13:02:25.607408 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.786257 (+178849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.907147 (+1120890us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:26.907379 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359964032}
I20260709 13:02:26.828217 15111 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.939303 15207 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.939891 15004 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:26.911517 15115 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.940249  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1108) took 1285 ms. Trace:
I20260709 13:02:26.848026 15147 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:26.940274 15111 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.862960 15168 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.921485 14997 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:26.922284 15117 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:26.946004  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2626) took 1914 ms (client timeout 2027 ms). Trace:
I20260709 13:02:26.946791 15117 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:26.924355 15214 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:26.947795  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:26.880940  8743 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:26.948611 15111 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.949002 15111 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:26.927189  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.949369 15111 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.914314 14923 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:26.949813  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2825) took 2160 ms. Trace:
I20260709 13:02:26.950294  8897 rpcz_store.cc:276] 0709 13:02:24.789407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.789512 (+   105us) service_pool.cc:224] Handling call
0709 13:02:24.887065 (+ 97553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.560800 (+673735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.949448 (+1388648us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540019ebd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:26.949724 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400959872}
I20260709 13:02:26.951009 15157 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.929082 15217 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:26.952095 15217 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.952325 15051 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:26.831822 14615 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.927791  9440 rpcz_store.cc:276] 0709 13:02:25.588256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.589141 (+   885us) service_pool.cc:224] Handling call
0709 13:02:26.896931 (+1307790us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400663ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:26.897108 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112092928}
I20260709 13:02:26.949569  8824 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
W20260709 13:02:26.946491  8898 rpcz_store.cc:269] 0709 13:02:25.031748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.032813 (+  1065us) service_pool.cc:224] Handling call
0709 13:02:26.945758 (+1912945us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:26.945934 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422360448}
I20260709 13:02:26.940825  8892 rpcz_store.cc:276] 0709 13:02:25.655253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.656063 (+   810us) service_pool.cc:224] Handling call
0709 13:02:25.656428 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.271873 (+615445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.939892 (+668019us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540032ad54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:26.940143 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403830656}
I20260709 13:02:26.863678 14796 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.965705  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.968253  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:26.965512  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.973791 14796 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:26.970275 15076 consensus_meta.cc:287] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.001s
I20260709 13:02:26.974472  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.974709 15076 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.976821 15076 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:26.975069  8815 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.982901  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:26.995313 15157 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.995610 15068 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.996012 15157 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.003290  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:26.976877 15212 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:26.940663 15138 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:26.967059 15218 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:27.025247  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.027910  8820 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:27.035611  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.036453  8740 raft_consensus.cc:2368] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15.
I20260709 13:02:27.042105  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.042827  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:27.047664  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 78169.5us
I20260709 13:02:26.979339 14992 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:27.046849  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 298731us
I20260709 13:02:27.042109  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.054029  8743 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.055815  8823 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 15.
I20260709 13:02:27.060091  8740 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:02:27.067173  8862 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:27.068653  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 258682us
I20260709 13:02:27.072294  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.070976  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:27.076272  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.078758 15228 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.079324 15228 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.084636  8815 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:27.092474  8740 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:27.094076  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.095741  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:27.075788 15151 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:27.102285  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:27.104072  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 43845.3us
I20260709 13:02:27.105162  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.085438 15218 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.107189 14923 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:27.107774  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49120.2us
I20260709 13:02:27.109433 14992 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:27.110728 15233 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:02:27.114269  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.115696 15076 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:27.115885  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.116566  8744 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:27.117146  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.112943  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.115972  8863 leader_election.cc:400] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15.
I20260709 13:02:27.118259  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.118705  8863 leader_election.cc:403] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:27.123371 14992 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:27.123974 14992 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:27.124881 14992 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.125633 14821 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.127408 15157 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.093051 15228 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.092067 14950 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:27.129441  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.138512 15161 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:27.141673  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52479.1us
I20260709 13:02:27.143114 15169 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.145478 14950 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:27.146096  8741 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:27.146649  8740 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:27.147150  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.129626  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.122776 14923 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:27.148646  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:27.137566  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.161056  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.160271  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.171329  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.096886 15115 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:27.183008  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.185495  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.189433  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.190706  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.169210 15168 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:27.192986  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.192333 15134 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.196033  9442 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:27.198920  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.200636  8741 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.201335  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.201949  9440 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:27.202730  9445 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:27.203859  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 83160.2us
I20260709 13:02:27.204469  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.205432 14950 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:27.206400 15206 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.720s	user 0.010s	sys 0.001s
I20260709 13:02:27.206842 15206 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.203872  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:27.209667  8742 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:27.213745 15169 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15.
I20260709 13:02:27.213987  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.214468  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.215279  9443 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:27.215862 15111 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:27.218063  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 126163us
I20260709 13:02:27.219364  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.220404 14923 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.221027 15207 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.221247  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.219427 15111 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:27.222606 15207 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.222743 15247 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.188357 14615 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.232259  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.233088 15147 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:27.188508 14839 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.232244 15173 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:27.173509 15031 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:27.188257 15236 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.256685 15173 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:27.199573 15168 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.257135 15173 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:27.257365 15168 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:27.258105 15173 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.259322 15173 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.260468  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.261235  8742 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:27.261996  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.262822  8898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56790
I20260709 13:02:27.263360  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.203550 15161 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:27.264045  8742 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:27.235158  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.266588 15031 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.262012  8749 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:27.270525  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.268680  9415 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:27.245437 15054 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.272040 15234 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:02:27.272253 15054 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.273010 15234 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:27.246476  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2824) took 2457 ms. Trace:
I20260709 13:02:27.249754 14917 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:27.273635 14917 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:27.253837  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.274430  8744 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:27.274698 14917 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.275985 14917 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.276268 15054 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.254354 15138 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.273504  8896 rpcz_store.cc:276] 0709 13:02:24.788892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.789056 (+   164us) service_pool.cc:224] Handling call
0709 13:02:25.851829 (+1062773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.913654 (+ 61825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.246251 (+1332597us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540046da50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:27.246420 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386681088}
I20260709 13:02:27.243418  8743 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:02:27.277853 15248 log.cc:927] Time spent T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.059s	user 0.002s	sys 0.001s
I20260709 13:02:27.263502  8898 raft_consensus.cc:606] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:27.278666  8898 raft_consensus.cc:614] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:27.277880  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.209435  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.279763  9447 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:27.280328 15001 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:27.280643 15134 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:27.281395 15134 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.280619  8816 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:27.264083 15161 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.283602  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.283990 15161 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.284403  8745 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:27.235131  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.286154  8742 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:27.286504  8747 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:27.287321  8742 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:27.287842  8747 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:27.281785 15134 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:27.278507  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.242904 14950 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.289182 14615 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.290171  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.273414 15234 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:27.290798  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2297) took 1571 ms. Trace:
I20260709 13:02:27.233276 15076 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.292551 15206 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.271577  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.234222 15147 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:27.294463 15116 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.280755 15001 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:27.306770  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 102974us
W20260709 13:02:27.306841  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66812.2us
I20260709 13:02:27.303635  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.311347  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.323148  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.324517  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.313702  8741 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:27.316931  9443 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:27.319017  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.325238  8815 raft_consensus.cc:2368] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15.
I20260709 13:02:27.316249 14796 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:02:27.327596  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:27.328431  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:27.327697  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:27.330503  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.297315 14862 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.329066  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:27.334513  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:27.334861  8894 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:27.337137  8894 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:27.310001 15123 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:27.310271 15169 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:27.338296  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.310297  8748 rpcz_store.cc:276] 0709 13:02:25.719294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.722995 (+  3701us) service_pool.cc:224] Handling call
0709 13:02:25.723620 (+   625us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:25.805133 (+ 81513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.805185 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:25.935027 (+129842us) log.cc:844] Serialized 52 byte log entry
0709 13:02:26.248914 (+313887us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:26.369998 (+121084us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.750287 (+380289us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.892501 (+142214us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b1800344620. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f64c7060 00007f90f64d9716 00007f90f64ea287 00007f90f64ea208
0709 13:02:27.089638 (+197137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.290319 (+200681us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b54001d9054. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:27.290368 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.290406 (+    38us) raft_consensus.cc:1685] finished
0709 13:02:27.290417 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.290542 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110999,"prepare.queue_time_us":266280,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":194889,"raft.run_cpu_time_us":3348,"raft.run_wall_time_us":102110,"spinlock_wait_cycles":106458240,"thread_start_us":791103,"threads_started":4,"wal-append.queue_time_us":348954}
I20260709 13:02:27.338317  8747 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:27.340791  8893 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 15.
I20260709 13:02:27.332408  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:27.331638  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:27.342239  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:27.342408  8897 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:27.343446  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.343573  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.344008  8818 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:27.347189  8743 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:27.323488 15147 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:27.353797  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.345703  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.369761  8896 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:27.319620 15212 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:27.373231  8744 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:27.375972  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:27.378556  8818 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:27.379544  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.342511 15123 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:27.386124  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.386272  9440 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:27.349706 15115 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:27.386766  8818 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:27.329489 14615 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.393525  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.391616  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.396238  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.334590 15116 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.397066  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.397207  9440 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:27.397590  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 81747.3us
I20260709 13:02:27.389691 15151 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.389657 15133 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:27.399391  8822 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
W20260709 13:02:27.389743 15098 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.000s
I20260709 13:02:27.399495  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:27.399768  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.419813  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.390940 15217 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.384920 14571 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.400983  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.422830  9445 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:27.423122  8742 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:27.406543  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.395821 15142 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:02:27.394120 15145 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:02:27.394958 15001 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.383517 15031 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:02:27.439947  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 108356us
I20260709 13:02:27.440537  8898 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:27.444120  9447 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:02:27.446254  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:27.447301  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.450830  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.451762  8822 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 13: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:27.461555  8821 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:27.466993  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.465087  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:27.469651  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78687.6us
I20260709 13:02:27.450090 14571 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.449532 15142 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:27.396946 15115 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.500474 15142 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:27.506570 15145 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:27.506973 15145 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:27.507201 15142 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.507766 15145 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.508744 15142 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.508915 15145 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.510303 15235 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.510825 15115 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.514797  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:27.449110 15123 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.396720 15234 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.526033  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.504329 14894 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.516602 15108 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.560860  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110500us
W20260709 13:02:27.577878  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 137123us
I20260709 13:02:27.606686  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.607669  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.505364 15168 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.500023 15169 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:27.614131  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.610075  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.610491  9448 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:27.609722  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.611716  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:27.518877 15212 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
W20260709 13:02:27.618060  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59871.2us
I20260709 13:02:27.517532 14571 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.621459  9445 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:27.580942 15117 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.613229 15214 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:27.525563 15250 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:02:27.630024  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:27.582218 15123 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.642335  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:27.643864  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:27.568114 15115 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:02:27.612483 14862 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.584069 15108 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:27.655680  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76794.8us
I20260709 13:02:27.641753 14894 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.658329  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.648715 15234 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:27.671509  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 228332us
I20260709 13:02:27.646925 15133 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:27.654299 15098 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:27.632165 15257 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:02:27.689587  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:27.695490  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.646301 15117 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.673209 14615 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.672057 15264 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.677086 15249 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
W20260709 13:02:27.702653  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.682873 15108 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.714664  8897 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:27.678548 15098 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:27.674098 14821 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:02:27.684456 15116 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.682008 15268 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.715749  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:27.725445  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 162678us
I20260709 13:02:27.676599 15256 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:02:27.730981  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.718848 14615 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:27.716871 15111 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:27.720230 15098 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.719254 15232 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.720438 14923 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.703207 15250 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:02:27.715533 15271 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:27.768597 15145 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.731503 15263 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.692792 15169 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.753321 15257 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.738009 15266 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:27.768321  8749 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:27.775633  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.714174 15264 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:27.782967  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67369.5us
I20260709 13:02:27.788502  9440 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:27.779505 15271 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:27.756106 14615 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.795866 15283 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:27.797886 15111 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.790959 15168 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:27.811128  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 136693us
I20260709 13:02:27.771574 15232 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.758740 15268 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.781682 15263 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.815279  8864 leader_election.cc:400] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15.
W20260709 13:02:27.801705  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.844013  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.792856 15266 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:27.783927 15257 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:27.818931 15269 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:02:27.819687 15270 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:27.820113 14615 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:27.820379 15251 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:02:27.856043 15260 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:02:27.820587 15261 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:02:27.778971 15108 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:02:27.819185 15271 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.857661  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1112) took 2201 ms. Trace:
I20260709 13:02:27.858049  8895 rpcz_store.cc:276] 0709 13:02:25.656494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.659257 (+  2763us) service_pool.cc:224] Handling call
0709 13:02:27.212194 (+1552937us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b58001e2498. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aae9c 00007f90f33aaa5b 00007f90fc2c3c09 00007f90fb41dee0 00007f90fb4e70ab 00007f90fb4e7131 00007f90fb4a8fc0 00007f90fb493929 00007f90fb4933d2 00007f90f4a0abd4
0709 13:02:27.212422 (+   228us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:27.241576 (+ 29154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.241616 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:27.242271 (+   655us) log.cc:844] Serialized 52 byte log entry
0709 13:02:27.378171 (+135900us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:27.378230 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.378419 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.857354 (+478935us) raft_consensus.cc:1685] finished
0709 13:02:27.857375 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.857600 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87771,"raft.queue_time_us":29842,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":31971584,"thread_start_us":135472,"threads_started":1,"wal-append.queue_time_us":203751}
I20260709 13:02:27.859539 15263 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.859844 15004 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.822299 15280 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.822863 15277 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.861269 15004 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.863746 15217 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.861518 15231 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:27.861796 15281 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:27.865810 15266 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:27.824599  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.825431  8787 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:27.862418 15277 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.737841 15133 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.839457 15151 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.833873 15256 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:27.830077 15283 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:27.845737  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:27.841267 15173 consensus_meta.cc:287] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.582s	user 0.014s	sys 0.002s
I20260709 13:02:27.842708 14571 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:27.866389 15281 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:27.823374 14917 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
W20260709 13:02:27.877465  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 52179us
I20260709 13:02:27.865847 15231 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:27.890861 15257 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.891640 15277 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:27.892036 15276 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:27.862080 15280 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.892449 15098 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.894714 15280 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:27.897562 15098 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.897457 15279 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:27.898255 15242 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.680s	user 0.003s	sys 0.001s
I20260709 13:02:27.898448 15279 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.900714 15268 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:27.901010 15279 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:27.896538 15281 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.902069 15283 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.877923 14894 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.814544 15076 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:27.903759  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58129.3us
I20260709 13:02:27.910765  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.903275 15281 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.917441 15285 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.920447 15264 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.921089 15283 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:27.922767  9441 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:02:27.890933 15231 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.932884 14917 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.933265 15232 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:27.933434  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:27.934666  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.934849 15159 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:27.935284 15159 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.935629 15159 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.935720  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:27.936363 15134 consensus_meta.cc:287] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.611s	user 0.003s	sys 0.007s
I20260709 13:02:27.936578 15264 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:27.937837 14917 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:27.938026 15264 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:27.938297 14917 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:27.940304  8864 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:27.936787 15134 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.940979 14917 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.947276 15051 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:02:27.945449  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 502052us
W20260709 13:02:27.954381  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.955662  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.958108 15217 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.958608 15217 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:27.960798 15217 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:27.945796 15117 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:27.946899  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.964816  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.962886  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.961743 15225 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.958647 15159 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:27.965607  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:27.982070  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.982229  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.963979 15283 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.985510  8748 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:27.986788  8749 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:27.996238  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.996513  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.997056  8740 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:27.996310  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:27.997931  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.001039  8747 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:28.001334  8745 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:27.999385  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.003204  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.004703  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.006094  8746 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:27.984549 15285 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:27.984717 15051 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.963888 15217 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:02:28.015849  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:27.984850 15228 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:02:27.998428 15206 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:28.025648  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.006875 15111 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:28.027520  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.007215 15161 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.028366  8863 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.031863  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.035857  8820 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:02:28.038269  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 323594us
I20260709 13:02:28.018953 15285 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:28.044106  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.040318  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.031795 15256 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.030109 15111 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.048318  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 404537us
I20260709 13:02:28.019526 15142 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:02:28.032683 15161 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:28.038431 15123 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:28.053064  8788 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:28.041167 14862 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.062233  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.062853  8895 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:28.071586  8897 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:28.071704 15268 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.072384 15268 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.073514 15264 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:28.074123 15264 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.076190 15264 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:28.078083 15268 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.079612 15277 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:28.079990 15277 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.080322 15277 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.080732 15266 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.081346 15266 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.083606 15266 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:28.084280  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.085618  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.086380  8820 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:28.091276 15277 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.094664 15277 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.097637 15117 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:28.098227 15117 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:28.098683 15117 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:28.101342 15009 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.102190 15281 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:28.102840 15281 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:28.103940 15281 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.055893 15283 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.107213 15283 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.113965 15134 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:28.030408 15206 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.119860 15206 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:28.120570 15206 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.056275 15161 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.027254 15051 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.125331 15266 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.126121 15266 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.127460 15051 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.128796 15266 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:28.133137 15285 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.055114 15111 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.139056 15142 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:28.139492 15142 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.140358 15142 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.142724 15108 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:28.053386 15256 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.145052  8895 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:02:28.056295 15235 consensus_meta.cc:287] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.545s	user 0.000s	sys 0.015s
I20260709 13:02:28.144920  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.145691 15235 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.149224  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:28.150745 15245 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.930s	user 0.005s	sys 0.000s
I20260709 13:02:28.145272 15256 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.155435 15231 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:28.161352  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.164549 15271 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.165338 15271 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.171181 15031 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:28.171725 15031 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.173966 15271 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.174082 15147 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.175603 15147 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:28.183233  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.183904  9444 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:28.183255  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:28.193840  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:28.196987  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 139338us
I20260709 13:02:28.198035  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:28.201447  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.202325  8815 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:28.203192  9446 raft_consensus.cc:2368] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21.
I20260709 13:02:28.203744  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.203961  9445 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:28.204339  9443 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:28.121011 15206 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.204954  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.206818  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.207597  8815 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:28.207762  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.206327  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:28.208196  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.209650  8818 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:02:28.181915 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.019s	sys 0.044s
W20260709 13:02:28.214813 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.020s	sys 0.045s
W20260709 13:02:28.216835  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.221735  9442 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:28.231905  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.233253  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.177213 14796 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.236063  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.248042  8898 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.248309  8895 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.172755 15031 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.239590  8749 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:28.250355  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:28.271940  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 65814.7us
I20260709 13:02:28.273566  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.274433  8893 raft_consensus.cc:2393] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:28.255972  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.275460  8897 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.258141  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.276916  8890 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.261608  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.278334  8822 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:28.266286  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.266530  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.266748  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.266933  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.235742 15271 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:28.221426 15206 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.267006  8895 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:02:28.267166  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.286232  8744 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:28.268967  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.267397  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:28.267992  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 215806us
I20260709 13:02:28.257856  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.266933  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.287753  8894 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.287202  8747 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:28.258376  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.289587  8746 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:28.267320  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.285949  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.290557  8892 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:28.247618 14796 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.274207  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.295789  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.292114  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.296926  8889 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:02:28.292568  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.292505  8742 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:28.298306  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:28.298350  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 270979us
I20260709 13:02:28.298708  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.257886 15142 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.304052  8789 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.301493  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.304579  8898 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:28.250062 14917 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.309068  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.329484  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:28.304160  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:28.332172  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.292789 15031 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.276580 15235 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.284687 15283 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.333447  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:28.335307  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.343253  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43542.5us
I20260709 13:02:28.300899 14796 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.292564 15271 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.300277 14950 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:28.306931 15277 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:28.341707  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.340057  8788 leader_election.cc:400] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21.
I20260709 13:02:28.408998  8788 leader_election.cc:403] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:28.340804 15134 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:28.340252 15235 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.342150 15108 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.337183  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.343468 14923 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:28.334102 15145 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.762857  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:28.329344 14862 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
W20260709 13:02:28.436481  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 97348.3us
I20260709 13:02:28.352720  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.330024 15159 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.361052 15001 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.331393 15280 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.267767 15111 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.371104  8891 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
W20260709 13:02:28.379984  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.383985  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:28.339757 15290 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:02:28.386355  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.310273 14821 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:28.393060  9447 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:28.398434  8890 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:28.398670  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 94723.9us
I20260709 13:02:28.409085  8889 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:28.432392  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55581.4us
I20260709 13:02:28.408845 15277 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.407692 15271 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:28.443102  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.448541  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.459789  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:28.453641  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 56242.7us
I20260709 13:02:28.453826  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.409322 14950 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.455433  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.461278  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.433486 15235 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:28.456452  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.456269  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:28.470005  8818 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.437932 15134 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.456146  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.463754  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.429297 15236 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 13:02:28.467291  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.479136  8893 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:28.467841  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.394637 15256 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.438611 15108 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.444094 15281 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:28.457309  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 226572us
W20260709 13:02:28.492552  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.476043  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 92159.8us
I20260709 13:02:28.476763  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.477466  8817 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:02:28.478343  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.481720  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.468482 15076 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.458976 15277 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.447293 14923 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.478312  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.453579 14821 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:28.491252  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.502530  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.504138  8890 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.468307 14950 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.480151  8894 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:28.468042 15289 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:28.496613  8823 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:02:28.510130  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.477695 15231 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.510699  8787 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.515450  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.516021  9441 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:28.518535  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69683.4us
I20260709 13:02:28.519402  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.519402  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.523867  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.488061 15280 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:28.528342  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.528723  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.536592  9444 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:28.530405  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.528923  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.537516  9445 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:28.529902  8817 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:28.531831  8889 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:28.493791 15108 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.541728  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.501674 15285 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.477921 15232 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:02:28.496464 15116 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:28.509595 14821 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:28.550359  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.550588 15304 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.550410  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.553432  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.554271  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.555066  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.556209  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.613572  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.565718 14917 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:28.524576 15250 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.539623  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.566967  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:28.621870  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 82367.6us
I20260709 13:02:28.622007  8897 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:28.570230  8892 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:28.569732 14923 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.618485 15009 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.634358 15009 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.635210 15247 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:28.635941 15247 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.637104 15247 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.638578  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:28.639789 15134 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.642230 15225 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:28.642767 15225 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:28.572127 15307 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:28.643723 15307 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.645823 15307 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:28.646485  9449 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:28.647825 15147 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:28.651244  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:28.525089 15123 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:28.577694 15309 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:02:28.577930 15300 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:02:28.583977  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.607173 15287 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.585759 15283 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:28.586719 15236 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:28.588773 15292 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.473338 15305 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:02:28.489902 15145 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:02:28.523244 15231 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:28.596705  8818 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:28.674573  8745 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:28.515049 15168 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.479624 15161 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.606542  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.561503  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.683326  8891 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.614383  8889 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:28.617408  8742 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:28.684006  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 164718us
W20260709 13:02:28.617558  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 67240.1us
I20260709 13:02:28.615027  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:28.623728  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.684996  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.624622  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:28.686911  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62434.5us
I20260709 13:02:28.687757  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:28.625439  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 82252.9us
I20260709 13:02:28.535699 15280 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:28.629721  8746 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:28.630880  9446 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:28.549558 15217 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:02:28.670462  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.612884 15290 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.597010  9443 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:28.678902  9442 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:28.684880  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:28.631775  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:28.705971  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.698722  8893 raft_consensus.cc:2368] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21.
I20260709 13:02:28.652158 15303 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:28.679896  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 209268us
I20260709 13:02:28.707252  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.726714  8745 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.672875 15307 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.709647  8748 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.714262  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.722251  8742 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:28.730105  8895 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.610548 15285 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.727304  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:28.734395  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49624.8us
W20260709 13:02:28.706480 15117 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
I20260709 13:02:28.744509  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.747623  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.679298 15231 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.748739  8889 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:28.752559  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:28.698249 14917 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.704890 15280 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:28.671396 15123 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.705621 15290 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.764101  9415 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.780407 15303 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.788986 15031 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.704270 15076 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.796468 15076 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.796900 15076 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.797483  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.799242 15161 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:28.800365 15314 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:02:28.805936 15157 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:28.806483 15157 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:28.807503 15157 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.808933 15157 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:28.723819 15250 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:28.778278  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.785631  8817 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:02:28.879331  8742 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:28.879957  8745 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:28.742797 15307 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.676528 15287 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.729444 15054 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.778012 15256 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:28.810144 15111 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:28.812899 15281 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21.
I20260709 13:02:28.813630 15322 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:28.815018 15285 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:28.752854 15228 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:28.820950  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.690487 15292 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.918835 15307 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.749166 15169 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.753190 15117 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.826496 15123 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:28.957931  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.826637  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.948928  8746 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:28.930413 15322 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.828486 15001 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.831053  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.963336  8896 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.834211  8744 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:28.777917  9446 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:28.837466 14917 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.839444 15116 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.755155 15206 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.846588  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:28.968289  8741 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.968366  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:28.852250  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.969467  8895 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:28.919359 14796 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:28.889707  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:28.891280  8747 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 13:02:28.972990  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:28.973529  8747 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:28.974081  8820 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:28.881168  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.923782  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:28.928174  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.921797 15235 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.976153  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:28.948853 15266 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:28.877539 15145 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.764024 15133 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:02:28.877002 15250 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:28.942571  8890 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:28.920082 15217 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:28.949915  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 205566us
I20260709 13:02:28.827436 15290 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.918605 15031 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:28.965101  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:28.965402 15322 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:28.937340 15305 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:28.964679 15001 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.938517  8740 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:28.976223  8822 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
W20260709 13:02:28.978093  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.005587  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.006413  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.981266  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.919180 14950 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.982951  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:29.010006  8895 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:02:28.985484  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:28.986110  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:28.991086  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:28.970016 15142 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.996001  8897 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:02:28.871908  8748 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:28.970804 15169 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.997860  8898 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:28.972765 15123 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:29.013146  8712 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:29.016642  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:29.022774  8740 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 13.
I20260709 13:02:29.023627  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:02:29.023963  8745 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 7
I20260709 13:02:29.024176  8745 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:29.018529  8895 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:29.025620  8895 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:29.021713  8748 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:29.024389  8745 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:29.025655  8741 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:29.027163  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.036069  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:29.038748  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:28.972530 15116 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:28.979656 14917 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.042222  8742 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:29.040822  8897 raft_consensus.cc:2410] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:29.021214 15031 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:28.999655 15271 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.066413  8815 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 13.
I20260709 13:02:29.067649  8817 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:28.996191 15217 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.015084 15236 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.992805 15145 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.014796 15285 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:02:29.027334 15142 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.017372 15305 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.074158  8816 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:29.036755 15277 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.089746  8818 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:29.056283 15134 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:02:29.066226 15332 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.066838 15031 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:29.043048 15123 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.089386  8741 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:29.046433 15169 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.051735 14821 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.089362 15228 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:29.141563 15228 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:29.141914 15292 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:29.142472 15228 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.143045 14950 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.114063  9440 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:29.114776 15206 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.115989 15123 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.118628 15266 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:29.119534 15108 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.001s
W20260709 13:02:29.149408  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.120450 15319 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:29.122164 15001 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:29.123525 14917 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.124981 15271 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.127395 15277 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.128173  8894 raft_consensus.cc:2368] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:29.129307 15303 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:29.130648 15305 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.107235 15031 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:29.134136 15328 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:02:29.134800 15324 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:02:29.136251 15217 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.137162  9449 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:02:29.062759 15250 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.137472 15116 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.139072 14796 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:29.198577 15293 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:29.111598  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 72959.2us
I20260709 13:02:29.155893  8820 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:29.120355 15145 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.159781  8896 raft_consensus.cc:2410] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:02:29.163072  8817 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:29.205399  8816 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:29.145758  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:29.148674  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.140830 15231 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.113637 15332 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:29.137010 15157 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:02:29.198065 15266 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.146806  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.305605  8712 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:29.316293  8787 leader_election.cc:336] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 2.027s (SENT)
I20260709 13:02:29.355973  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:29.353673  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.357090  8816 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.427969  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:29.230069 15287 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:29.428501  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 123019us
I20260709 13:02:29.432339  9447 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:29.202399 15108 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.477113 15108 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.480466  8821 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:29.432191 15145 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 13:02:29.445199  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 129000us
I20260709 13:02:29.446089  8741 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:02:29.447542 15327 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 13:02:29.449091 15231 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:29.449577  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.511597  8823 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:29.450731  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.483635  8787 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 3.155s (SENT) [suppressed 21 similar messages]
I20260709 13:02:29.450805 15157 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:29.451454  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:29.517669  8742 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:29.453045  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:29.453557 15217 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.458582 15169 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.459036  8816 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:29.520532  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.521449  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
W20260709 13:02:29.459255 15076 consensus_meta.cc:287] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.005s
I20260709 13:02:29.355069 15307 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.465973 15332 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.466943  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2389) took 1015 ms. Trace:
W20260709 13:02:29.472034 15280 consensus_meta.cc:287] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.000s
I20260709 13:02:29.346933 15293 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.518950  8748 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:29.530622  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:29.531734  8741 raft_consensus.cc:2393] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:29.524121  8819 rpcz_store.cc:276] 0709 13:02:28.451690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.451871 (+   181us) service_pool.cc:224] Handling call
0709 13:02:28.452247 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.465421 (+ 13174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.465466 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.484780 (+ 19314us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.756345 (+271565us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:28.780530 (+ 24185us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.781775 (+  1245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.781851 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.466692 (+684841us) raft_consensus.cc:1685] finished
0709 13:02:29.466721 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.466886 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4616,"prepare.queue_time_us":25367,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":166179,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":28018,"thread_start_us":213805,"threads_started":1,"wal-append.queue_time_us":420746}
I20260709 13:02:29.485422 15250 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:29.540612  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.549846  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:29.551087  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:29.570286  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.574663  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:29.578408 15307 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.578802 15338 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:29.579203  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.579818  8816 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.581357 15206 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.581866 15206 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.584008 15206 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.584509 15305 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.584995 15305 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:29.586236 15345 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:29.584519  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:29.587199 15305 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.588617 15134 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:29.588809 15271 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:29.589371 15134 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:29.589334 15271 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.590029 15108 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:29.590111 15271 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.590555 15108 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.591022 15108 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:29.591464  8742 raft_consensus.cc:2468] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:29.530344 15157 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:29.592419  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:29.592984  8742 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.593241 15157 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.594388  8749 raft_consensus.cc:1240] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:29.530471 15217 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.599279 15217 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.519645 15340 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.601425  8742 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:29.601877 15340 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.523327 15318 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:29.604166 15322 consensus_meta.cc:287] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
I20260709 13:02:29.606477 15293 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:29.608821 15290 consensus_meta.cc:287] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.000s
I20260709 13:02:29.540496 15346 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.529958 15076 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.619074  8748 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:29.620802  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:29.627843  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:29.684572  8741 raft_consensus.cc:2410] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:02:29.685477  8742 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:29.636539  8823 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:29.655704  8816 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:02:29.659988  9446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.090s	sys 0.165s
I20260709 13:02:29.611475  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2867) took 1355 ms. Trace:
I20260709 13:02:29.670394  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.688079  8897 rpcz_store.cc:276] 0709 13:02:28.255737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.070096 (+814359us) service_pool.cc:224] Handling call
0709 13:02:29.070446 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.070818 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.070857 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.071414 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.229856 (+158442us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:29.350423 (+120567us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.428217 (+ 77794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.428289 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.611185 (+182896us) raft_consensus.cc:1685] finished
0709 13:02:29.611212 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.611402 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77480,"prepare.queue_time_us":120370,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":16976,"raft.queue_time_us":80084,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":52258,"thread_start_us":125303,"threads_started":1,"wal-append.queue_time_us":396797}
W20260709 13:02:29.691263  9446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.090s	sys 0.165s
I20260709 13:02:29.671284  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:29.677865  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 101797us
I20260709 13:02:29.694466  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.671456 15217 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.699900  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.700342  8897 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:29.708104  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.688315 15115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.068s	sys 0.118s
I20260709 13:02:29.720757  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1137) took 1402 ms. Trace:
I20260709 13:02:29.721961  8897 rpcz_store.cc:276] 0709 13:02:28.318518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.697130 (+1378612us) service_pool.cc:224] Handling call
0709 13:02:29.697460 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.720483 (+ 23023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.720703 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.536501 15280 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.721134  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.724525  8816 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:29.729434  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.729970  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2647) took 1365 ms. Trace:
I20260709 13:02:29.730340  8897 rpcz_store.cc:276] 0709 13:02:28.358603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.722941 (+1364338us) service_pool.cc:224] Handling call
0709 13:02:29.723292 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.723533 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.723566 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.723606 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:29.723638 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.723723 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.723887 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.699456 15338 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:02:29.717001 15284 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.732197  8897 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:29.739281  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2649) took 1380 ms. Trace:
I20260709 13:02:29.739531  8895 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:29.739622  8897 rpcz_store.cc:276] 0709 13:02:28.359140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.731264 (+1372124us) service_pool.cc:224] Handling call
0709 13:02:29.731543 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.739017 (+  7474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.739227 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.680402 15322 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:29.721908 15115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.068s	sys 0.118s
I20260709 13:02:29.730803  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.741098  8897 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:29.751636  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2650) took 1392 ms. Trace:
I20260709 13:02:29.742340  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2346) took 1128 ms. Trace:
I20260709 13:02:29.683563 15290 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.751986  8897 rpcz_store.cc:276] 0709 13:02:28.359369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.740567 (+1381198us) service_pool.cc:224] Handling call
0709 13:02:29.740933 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.751381 (+ 10448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.751579 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.752449  8743 rpcz_store.cc:276] 0709 13:02:28.613929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.614047 (+   118us) service_pool.cc:224] Handling call
0709 13:02:28.633672 (+ 19625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.652730 (+ 19058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.652758 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.652799 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:28.652817 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.742049 (+1089232us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54001d9a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:29.742116 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.742271 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43367680}
I20260709 13:02:29.753973 15290 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.754063  8897 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:29.727816 14821 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.754734  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2651) took 1395 ms. Trace:
I20260709 13:02:29.755249  8819 raft_consensus.cc:2368] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:02:29.752277 15147 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:02:29.755420  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.755663 15147 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:29.756067  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.756548 15147 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.757519  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.755055  8897 rpcz_store.cc:276] 0709 13:02:28.359611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.753598 (+1393987us) service_pool.cc:224] Handling call
0709 13:02:29.753891 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.754508 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.754686 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.758343 15340 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.758589  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.699571 15293 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:29.723869 15116 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:02:29.760506  8815 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 20. Ops: []
I20260709 13:02:29.757560  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.692016 15346 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.746889  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.758957  8897 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 17. Ops: [15.9-15.9]
I20260709 13:02:29.763015  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:29.767004  9441 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.759199  9447 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
W20260709 13:02:29.745849 15054 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.001s
I20260709 13:02:29.768796  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2654) took 1408 ms. Trace:
I20260709 13:02:29.757560 15147 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.779703  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:29.725018 15217 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.782249  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.783406  8897 rpcz_store.cc:276] 0709 13:02:28.360341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.758423 (+1398082us) service_pool.cc:224] Handling call
0709 13:02:29.758715 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.768488 (+  9773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.768731 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.800616  8897 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:29.801522  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2655) took 1436 ms. Trace:
I20260709 13:02:29.802850  8897 rpcz_store.cc:276] 0709 13:02:28.365065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.794389 (+1429324us) service_pool.cc:224] Handling call
0709 13:02:29.794792 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.801227 (+  6435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.801461 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.761346 15051 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.802351  8816 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:29.690817 15134 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.790782  9441 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:29.810211  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:29.807191  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 309174us
I20260709 13:02:29.744328 15331 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:02:29.813062  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2659) took 1440 ms. Trace:
I20260709 13:02:29.811124  9444 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:29.813553  8897 rpcz_store.cc:276] 0709 13:02:28.368235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.804147 (+1435912us) service_pool.cc:224] Handling call
0709 13:02:29.804534 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.804757 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.804795 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.807471 (+  2676us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:29.807508 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.807585 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.808093 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.792192 15054 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:29.784780 15169 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:29.785684 15340 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:29.818531  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.799371 15322 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.824270  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.828792  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.832970  8816 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:29.834062  8897 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:29.834956  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2873) took 1371 ms. Trace:
I20260709 13:02:29.835417  8897 rpcz_store.cc:276] 0709 13:02:28.463458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.814894 (+1351436us) service_pool.cc:224] Handling call
0709 13:02:29.815260 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.834631 (+ 19371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.834893 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.837308  8897 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:02:29.838083  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1138) took 1307 ms. Trace:
I20260709 13:02:29.838536  8897 rpcz_store.cc:276] 0709 13:02:28.530187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.836604 (+1306417us) service_pool.cc:224] Handling call
0709 13:02:29.837002 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.837796 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.838024 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.841266  8897 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:29.842059  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1140) took 1001 ms. Trace:
I20260709 13:02:29.842548  8897 rpcz_store.cc:276] 0709 13:02:28.840834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.840739 (+999905us) service_pool.cc:224] Handling call
0709 13:02:29.841037 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.841802 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.842002 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.844080  8897 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:02:29.844821  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1141) took 1003 ms. Trace:
I20260709 13:02:29.845257  8897 rpcz_store.cc:276] 0709 13:02:28.841192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.843438 (+1002246us) service_pool.cc:224] Handling call
0709 13:02:29.843806 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.844553 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.844765 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.846604  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58306 (request call id 1249) took 1340 ms. Trace:
I20260709 13:02:29.847056  9442 rpcz_store.cc:276] 0709 13:02:28.505950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.509377 (+  3427us) service_pool.cc:224] Handling call
0709 13:02:29.846398 (+1337021us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54006a8954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:29.846545 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323589376}
I20260709 13:02:29.847882  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:56804 (request call id 0) took 1043 ms. Trace:
I20260709 13:02:29.848371  8897 rpcz_store.cc:276] 0709 13:02:28.804297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.846444 (+1042147us) service_pool.cc:224] Handling call
0709 13:02:29.847776 (+  1332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:29.851516  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1173) took 1347 ms. Trace:
I20260709 13:02:29.851917  9440 rpcz_store.cc:276] 0709 13:02:28.504360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.504513 (+   153us) service_pool.cc:224] Handling call
0709 13:02:28.529712 (+ 25199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.153495 (+623783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.851279 (+697784us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b540059f6d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:29.851430 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343587072}
W20260709 13:02:29.856220  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 372681us
I20260709 13:02:29.827837 15340 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.813752 15051 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.827549 15169 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.882249  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.885275  8823 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.865617 15322 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:29.900441  8745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.030s	sys 0.050s
W20260709 13:02:29.900985  8745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.030s	sys 0.050s
W20260709 13:02:29.905927  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 271502us
I20260709 13:02:29.891156 15076 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.878532 15271 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.914835  8823 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:29.916298  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:29.901364 15280 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.905450 15284 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:29.921165 15169 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:29.922206 15284 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.923336 15271 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.934116  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2388) took 1491 ms. Trace:
I20260709 13:02:29.934661  8822 rpcz_store.cc:276] 0709 13:02:28.442639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.442752 (+   113us) service_pool.cc:224] Handling call
0709 13:02:28.468008 (+ 25256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.005317 (+537309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.933779 (+928462us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b5400076554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:29.934037 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168356096}
I20260709 13:02:29.938102  8822 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:29.945771 15142 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.946620 15142 consensus_queue.cc:1059] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:29.947613 15142 raft_consensus.cc:3055] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:29.947992 15142 raft_consensus.cc:740] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:29.949263  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 247898us
I20260709 13:02:29.950562  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:29.953646  8789 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:29.906167 15322 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.963059 15116 raft_consensus.cc:2804] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:29.961685 15271 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:29.966584 15116 raft_consensus.cc:697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:29.968039 15116 consensus_queue.cc:237] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:29.968870 15342 log.cc:927] Time spent T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:02:29.915271 15168 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:02:29.974016 15051 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:29.974357  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:29.976532  8820 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:29.976920  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2872) took 1719 ms. Trace:
I20260709 13:02:29.973935 15340 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:29.978013  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2387) took 1535 ms (client timeout 1897 ms). Trace:
W20260709 13:02:29.982882  8821 rpcz_store.cc:269] 0709 13:02:28.442310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.442762 (+   452us) service_pool.cc:224] Handling call
0709 13:02:29.977734 (+1534972us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540001fc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:29.977917 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347470080}
W20260709 13:02:29.980931  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.983892  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:29.982616  8892 rpcz_store.cc:276] 0709 13:02:28.257047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.581346 (+1324299us) service_pool.cc:224] Handling call
0709 13:02:29.581715 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:29.582118 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.582150 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:29.582753 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 13:02:29.857633 (+274880us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:29.863122 (+  5489us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.865969 (+  2847us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.876733 (+ 10764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.876836 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.976690 (+ 99854us) raft_consensus.cc:1685] finished
0709 13:02:29.976711 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.976863 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12615,"prepare.queue_time_us":30778,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":89,"raft.queue_time_us":20633,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":11744,"spinlock_wait_cycles":333056,"thread_start_us":274327,"threads_started":1,"wal-append.queue_time_us":380566}
I20260709 13:02:29.938906 15266 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:29.986483  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:29.989080  8749 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:29.989630 15357 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:29.990154 15357 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:29.991217 15357 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.992440  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:29.992743 15357 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:29.993547 15168 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:29.993956 15168 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.994369 15168 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:29.995299 14821 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.995946 14821 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:29.996726  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2354) took 1069 ms. Trace:
I20260709 13:02:29.997165  8745 rpcz_store.cc:276] 0709 13:02:28.927517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.037567 (+110050us) service_pool.cc:224] Handling call
0709 13:02:29.037819 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.088817 (+ 50998us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:29.345621 (+256804us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.903976 (+558355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.904850 (+   874us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:29.904920 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.996494 (+ 91574us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540013bfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:29.996544 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.996672 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":512384,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":40,"raft.queue_time_us":120,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3802,"spinlock_wait_cycles":117766784,"thread_start_us":813250,"threads_started":2,"wal-append.queue_time_us":647315}
I20260709 13:02:29.999338 14821 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.000046 14821 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.000870 14821 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.003257 15235 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.003619 15168 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.003856  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:30.004741  8890 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2662) took 1633 ms (client timeout 2031 ms). Trace:
W20260709 13:02:30.004024  8746 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2590) took 1240 ms (client timeout 1644 ms). Trace:
I20260709 13:02:29.971155 15346 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.007328  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2378) took 1808 ms. Trace:
I20260709 13:02:30.014810  8896 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:30.016813  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.016444  8824 rpcz_store.cc:276] 0709 13:02:28.199260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.199426 (+   166us) service_pool.cc:224] Handling call
0709 13:02:28.199848 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:28.269106 (+ 69258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.269153 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:28.282242 (+ 13089us) log.cc:844] Serialized 52 byte log entry
0709 13:02:28.471170 (+188928us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:28.823696 (+352526us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b1800355420. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf
0709 13:02:28.823902 (+   206us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.824154 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.824253 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.994259 (+1170006us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b1800355420. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:30.007071 (+ 12812us) raft_consensus.cc:1685] finished
0709 13:02:30.007100 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.007253 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":144393,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":191296,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":31099,"spinlock_wait_cycles":49799296,"thread_start_us":368409,"threads_started":2,"wal-append.queue_time_us":372113}
I20260709 13:02:30.019271  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.020058  8824 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:02:30.014942  8746 rpcz_store.cc:269] 0709 13:02:28.763634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.763910 (+   276us) service_pool.cc:224] Handling call
0709 13:02:30.003815 (+1239905us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540016e4d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.003940 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346630912}
W20260709 13:02:30.014865  8890 rpcz_store.cc:269] 0709 13:02:28.371719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.516617 (+144898us) service_pool.cc:224] Handling call
0709 13:02:30.004514 (+1487897us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54003bdb54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.004678 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766031744}
I20260709 13:02:30.026889  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.028874  8749 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:30.030725  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.031437  8749 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.999989 15232 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:30.012945 15340 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:30.035332  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.038448  8824 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:30.011513 15217 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.040164  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.040731  8745 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:30.042551 15284 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:30.043743  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1109) took 1262 ms. Trace:
I20260709 13:02:30.045867  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.054024  8823 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:30.047587  8746 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.048269  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.050348 15266 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.050488  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.051795 15168 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.069928 15168 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:30.070250 15117 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.071533  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2868) took 1815 ms. Trace:
I20260709 13:02:30.071889  8896 rpcz_store.cc:276] 0709 13:02:28.255991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.354433 (+1098442us) service_pool.cc:224] Handling call
0709 13:02:30.006085 (+651652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.017113 (+ 11028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.071339 (+ 54226us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54003bdb50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.071474 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498003328}
I20260709 13:02:29.964560 15142 consensus_queue.cc:260] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.074045 15142 raft_consensus.cc:3060] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:30.056828 15108 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.074939 15225 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.076032 15225 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:30.075932  8822 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:30.053189  8749 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:30.055099  8820 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:30.053840  8818 rpcz_store.cc:276] 0709 13:02:28.781040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.781932 (+   892us) service_pool.cc:224] Handling call
0709 13:02:28.782313 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.093989 (+311676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.043397 (+949408us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400075654. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:30.043603 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706644864}
I20260709 13:02:30.058017  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:30.052233 15340 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.058821 15151 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:30.039161 15232 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:30.055361 15354 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.063401 15009 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.059439 15355 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:02:30.066115  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2862) took 1811 ms. Trace:
I20260709 13:02:30.086478  8895 rpcz_store.cc:276] 0709 13:02:28.254435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.038949 (+784514us) service_pool.cc:224] Handling call
0709 13:02:29.039202 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.741817 (+702615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.065891 (+324074us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540019e1d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:30.066063 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610576640}
I20260709 13:02:30.083181  8821 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:30.087952 15355 consensus_queue.cc:1243] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:02:30.059566 15031 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.765s	user 0.011s	sys 0.000s
I20260709 13:02:30.014472 14821 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.095031  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.101179  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.102936  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.103736  8749 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:30.105087  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.083971 15340 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.104894  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.110634  9440 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:30.107043 15355 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:30.106772 15362 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:30.124260  8787 leader_election.cc:336] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Timed out: RequestConsensusVote RPC to 127.7.117.131:44123 timed out after 1.646s (SENT)
I20260709 13:02:30.115376 15168 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:30.083878 15151 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.098414 15031 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.131248  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.131822  8749 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:30.133814  8746 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:30.091111 15354 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.141352  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:30.128562 15168 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.145319  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:30.129532 15355 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:30.126747 15232 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.156123 15054 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:30.156817 15354 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:30.158385 14821 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:30.158731 15031 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:30.158885  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:30.159060  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.166604  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.167145  8816 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:30.167907  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:30.170671  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:30.159795 15250 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.001s
I20260709 13:02:30.171825 15250 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:30.172189 15250 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.172513 15250 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:30.174726  8749 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:30.130432 15362 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.161366 15236 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:30.179044 15362 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.160960 15168 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:30.161849  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.186057  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.187446  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.188180 15117 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:30.188977  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.189492  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.181149  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.164309  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:30.192467  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.193053  9441 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.193321  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:30.190925  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.197131  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.197754  9448 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:30.198827  8816 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:30.202796  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.203110  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.199731  9415 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:30.181036 15236 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.212580  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:30.206611  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60094.3us
W20260709 13:02:30.214506  9443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58300 (request call id 1171) took 1993 ms (client timeout 1994 ms). Trace:
I20260709 13:02:30.215394  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1113) took 1423 ms. Trace:
W20260709 13:02:30.215940  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.212208 15116 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.217545 15268 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.202167 15054 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:30.218254  8894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2656) took 1852 ms (client timeout 2027 ms). Trace:
I20260709 13:02:30.218078 15268 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:30.218703  8894 rpcz_store.cc:269] 0709 13:02:28.365362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.515188 (+149826us) service_pool.cc:224] Handling call
0709 13:02:30.218051 (+1702863us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.218196 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407209856}
I20260709 13:02:30.220350 15268 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.242045 15151 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.241714 15168 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.243168  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2394) took 1111 ms. Trace:
I20260709 13:02:30.243691  8819 rpcz_store.cc:276] 0709 13:02:29.134420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.699647 (+565227us) service_pool.cc:224] Handling call
0709 13:02:30.242933 (+543286us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b54002120d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.243087 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85171456}
I20260709 13:02:30.244642 15168 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:30.245702  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2339) took 1694 ms (client timeout 1791 ms). Trace:
W20260709 13:02:30.246126  8744 rpcz_store.cc:269] 0709 13:02:28.551482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.553147 (+  1665us) service_pool.cc:224] Handling call
0709 13:02:30.245518 (+1692371us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400073ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.245641 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508147968}
I20260709 13:02:30.215773  8815 rpcz_store.cc:276] 0709 13:02:28.792376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.069304 (+276928us) service_pool.cc:224] Handling call
0709 13:02:29.079084 (+  9780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.765267 (+686183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.215186 (+449919us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540012b454. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:30.215340 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567072640}
I20260709 13:02:30.252805  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:30.214921  9443 rpcz_store.cc:269] 0709 13:02:28.220917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.222138 (+  1221us) service_pool.cc:224] Handling call
0709 13:02:30.214345 (+1992207us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54003a2854. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.214452 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719771904}
I20260709 13:02:30.217119 15031 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:30.256080  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.256471  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.256930  9449 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:02:30.258294  8819 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:02:30.259750 15151 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:30.263389  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.195036 15117 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:30.222574  9441 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:30.266407  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.213095 15354 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:30.227649 15343 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.419s	user 0.000s	sys 0.003s
I20260709 13:02:30.233573 15355 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.259773  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:30.319242  8816 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:30.319618  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.223348  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.202026 15076 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.326131  8896 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:30.266256  8695 auto_rebalancer.cc:893] Removing replace marker failed: Replica 7fd110c312c14e17a2bbe6d453493cf2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:30.305852  8893 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:30.212162 15228 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:30.327838  8695 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:02:30.317358 15054 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:30.328219  8695 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:30.328169 15369 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:30.328954 15054 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.329641  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.329902 15354 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:30.330405  9447 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:30.330761 15354 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.331365  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.216372 15266 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:30.338778  8816 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:30.321491  9443 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:30.323338  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.316203  9448 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:30.348523  8892 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:30.324153  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.349431  8890 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:30.266296 15338 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:30.255950 15290 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:30.350698 15290 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:30.342193  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 139529us
I20260709 13:02:30.352459  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.353068  8890 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:30.353945  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.354928  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.356091  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.360858  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.361538  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.370713  8896 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:30.265502 15371 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:30.313654 15031 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:30.363411 15340 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.365721 15293 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:30.266877 15346 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.213774 15341 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
I20260709 13:02:30.269482 15280 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.396997  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.362015 15375 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:30.397871  8890 raft_consensus.cc:2410] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:02:30.372432  9448 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
W20260709 13:02:30.370801  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.398695  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.399775  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.400718  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.401643  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.401762  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.402304  8896 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.396787 15280 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.402976  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.404160  8889 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:30.382297  8897 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:02:30.404415 15280 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.317735 15234 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:30.405498 15346 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.408197 15346 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.408815 15346 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.400944 15031 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.334515 15228 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.390805  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.318678 15168 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:02:30.395421  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.402261  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.405679 15234 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.340911 15266 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:30.413273  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.335220 15076 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.335080 15235 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:02:30.341110 15236 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:30.424798  8897 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:30.392912  9447 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:30.422946  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 117999us
I20260709 13:02:30.425798  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.426345  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.433275  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:30.433557 15338 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:30.360903 15250 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.427899  8896 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:30.435912  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.417873 15147 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:30.418179 15228 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.439604  8894 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:30.441462  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.442085  8787 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.443238  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.427366 15266 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:30.449542  8748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2610) took 1002 ms. Trace:
I20260709 13:02:30.449949  8748 rpcz_store.cc:276] 0709 13:02:29.446995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.569241 (+122246us) service_pool.cc:224] Handling call
0709 13:02:29.618840 (+ 49599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.683406 (+ 64566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.449351 (+765945us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5400160dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.449487 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40686464}
I20260709 13:02:30.456283  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.456967  8889 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:30.460872  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 1114) took 1668 ms (client timeout 1707 ms). Trace:
W20260709 13:02:30.461493  8817 rpcz_store.cc:269] 0709 13:02:28.792667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.889385 (+ 96718us) service_pool.cc:224] Handling call
0709 13:02:30.460658 (+1571273us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b5400076a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.460795 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216760576}
I20260709 13:02:30.464046  8893 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:30.464594 15108 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.470886 15280 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.471453 15280 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.474156  9446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1115) took 1467 ms. Trace:
I20260709 13:02:30.474587  9446 rpcz_store.cc:276] 0709 13:02:29.007079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.007706 (+   627us) service_pool.cc:224] Handling call
0709 13:02:29.037352 (+ 29646us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:29.037735 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.037764 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:29.041073 (+  3309us) log.cc:844] Serialized 52 byte log entry
0709 13:02:29.691688 (+650615us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:29.696706 (+  5018us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.960444 (+263738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.960522 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.223006 (+262484us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:30.473871 (+250865us) raft_consensus.cc:1685] finished
0709 13:02:30.473966 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.474094 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":793,"prepare.queue_time_us":5122,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":60820,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":14202,"spinlock_wait_cycles":3307904,"thread_start_us":878991,"threads_started":2,"wal-append.queue_time_us":686457}
I20260709 13:02:30.434279 15142 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:30.476737 15250 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:30.476945 15142 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.477352 15142 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:30.477489 15380 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:30.478022 15380 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:30.478396 15380 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.484325 15054 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:30.485682 15357 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.486163 15357 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.416220 15134 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:02:30.492287  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.492980  8897 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:30.487892  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:30.488396  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:30.494791  8817 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:30.495672  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:30.489383  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.465070  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.487198  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.492252  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.497386  8894 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:30.498971  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.499194  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:30.499684  8889 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:30.500993  8893 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:30.504242  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61226us
I20260709 13:02:30.505247  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.506232  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.506938  8817 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:30.521721  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:30.446902 15271 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:30.524120  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:30.502341 15368 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:30.525068  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.555431  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.525755  8894 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:30.507956 15262 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:30.504837 15277 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:30.556634  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.509243 15051 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.533964  8817 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:30.535120  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2400) took 1339 ms. Trace:
I20260709 13:02:30.572065  8822 rpcz_store.cc:276] 0709 13:02:29.195579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.939115 (+743536us) service_pool.cc:224] Handling call
0709 13:02:29.939493 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.078849 (+139356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.534870 (+456021us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:30.535047 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313274752}
W20260709 13:02:30.535849  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2863) took 2281 ms (client timeout 2049 ms). Trace:
W20260709 13:02:30.573259  8898 rpcz_store.cc:269] 0709 13:02:28.254716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.333095 (+ 78379us) service_pool.cc:224] Handling call
0709 13:02:30.535627 (+2202532us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b54000483d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:30.535748 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1503581056}
I20260709 13:02:30.574308  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.566920  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.575004  8898 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:30.537358 15206 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:30.537737  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.537894  8891 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:02:30.538434  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.526721 15271 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:30.539059  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.537272  9446 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:30.539083 15340 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.547339  8893 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:30.560158  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.560359  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:30.597854  8816 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:30.561009  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.508373 15145 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.555003  9415 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:30.598765  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43837.2us
I20260709 13:02:30.567315  8889 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:30.596012  8893 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:30.567880  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:30.530761 15280 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.569926  8745 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:30.526037 15169 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:30.578496  9444 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:30.585721  8898 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:30.581164  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.581812  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
W20260709 13:02:30.583278  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44311.8us
I20260709 13:02:30.611943  8898 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:30.555442 15236 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:30.602909  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.629492  8893 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:30.594970 15271 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.604530  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.599826 15217 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:30.537125 15380 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.566143 15262 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.615432  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:30.639276  9448 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.579547 15289 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:30.583244 15054 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:30.629904  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:30.605623  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.632915  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:30.601573  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:30.633967  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91238.3us
I20260709 13:02:30.635471  8897 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:30.578256 15250 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:02:30.614063 15284 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:30.613860 15117 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:30.610391 15231 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:30.697046  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.697922  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:30.714151  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 112673us
I20260709 13:02:30.714763  8744 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:30.715834  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.716408  8744 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:30.698549  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.717741  8749 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:30.728845  8749 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:30.613430 15280 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:30.699152  9448 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:30.731143  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.732551  8741 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:30.733326  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:30.703814 15262 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:30.747457  8747 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:30.636055 15217 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:30.752676  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 107929us
I20260709 13:02:30.752765 15217 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:30.753417  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:30.754444  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:30.736799  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:30.701886 15206 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:30.711791 15322 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:30.609468 15145 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.735070  8743 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:30.688670 15168 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:30.711951 15009 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.637393 15331 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:02:30.738461  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:30.739852  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:30.741003 15054 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:30.762223 15169 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:30.770180 15277 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:30.741202 15362 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:02:30.770345 15168 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.751755 15280 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.805222  9442 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:30.770475  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 194373us
I20260709 13:02:30.762019 15076 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:30.774899 15322 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:30.771862 15234 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:30.832134  8863 connection.cc:256] RPC call timeout handler was delayed by -1.0108177609999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:02:30.803275 15169 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.766983 15206 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.599987 15384 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 13:02:30.803709 15054 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:30.888406 15341 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.809453 15117 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:30.878571 15372 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 13:02:30.919502  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:30.920692  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 184019us
I20260709 13:02:30.786607 15009 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:30.888319 15250 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:30.916590 15322 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.895313  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:30.878180 15392 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:30.877974 15395 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:30.923945 15382 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 13:02:30.938242  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:30.948546  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 194186us
W20260709 13:02:30.932894 15379 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:02:30.925405 15284 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:31.044740  8864 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:31.051698  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:31.033466  8712 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.7.117.190:45111, user_credentials={real_user=slave}} blocked reactor thread for 81014.2us
I20260709 13:02:30.933952 15231 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:31.034701 15346 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:31.048684  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 110686us
I20260709 13:02:31.051612  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:31.045723  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 98040.2us
W20260709 13:02:31.057631  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.058368  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 155086us
I20260709 13:02:31.059840  9414 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:30.953101 15250 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:31.035230 15322 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.038664 15395 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.040012 15076 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.071018  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.047030 15396 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.075865  8749 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:31.047621 15376 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:31.046864 15392 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:30.949939 15341 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.062659 15031 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:31.054644 15346 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:31.069001 15250 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.090581  8822 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:31.069348 15380 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:31.078740 15383 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
I20260709 13:02:31.071779 15108 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:31.098943  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.114380  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.086225 15076 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.093344 15031 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.043478 15411 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:02:31.139945 15076 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:31.142010 15388 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
W20260709 13:02:31.142972 15391 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
W20260709 13:02:31.143532 15394 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:02:31.143777 15346 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.141130 15076 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:31.146191 15076 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:31.146705 15076 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.147576 15076 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.075385 15395 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.149718 15409 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.048455 15404 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:02:31.153692 15404 consensus_queue.cc:1243] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:31.078388 14923 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:31.089715 15168 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:31.091634 15400 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.137104 15232 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:31.137722 15341 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.138181 15031 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.138500 14821 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:31.148530 15206 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.114066 15392 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.158540  8712 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:31.066151 15277 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:31.159214 15142 consensus_meta.cc:287] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.004s
W20260709 13:02:31.181780  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 135047us
I20260709 13:02:31.199591  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.199745  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.165093 15145 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.088171 15376 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:31.203380  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 151754us
I20260709 13:02:31.205097 15414 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:31.205821 15409 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:31.206001  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:31.164465 14923 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:31.087627 15396 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:31.196834 15228 consensus_meta.cc:287] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.749s	user 0.006s	sys 0.005s
W20260709 13:02:31.195720 15266 consensus_meta.cc:287] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.000s
I20260709 13:02:31.211862  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.097447 15117 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.100159 15231 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.197844 15280 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.179831 14821 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:31.214411  8864 leader_election.cc:400] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:02:31.202369 15332 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:31.206869 15142 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.209537 15206 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.169268 15395 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.181329 15271 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.162781 15284 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:31.229616 15398 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:31.216126 15151 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:02:31.242101 15423 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:02:31.246860 15393 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:02:31.247339 15399 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:31.294668 15420 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:31.247556 15396 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.240119 15341 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.225529 15228 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.169570 15400 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.257647 15409 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:31.214121 15380 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:02:31.261606  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50320.5us
I20260709 13:02:31.275224  8864 leader_election.cc:403] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:31.274086  9442 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:31.227057 15266 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:31.274693 15217 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.006s
W20260709 13:02:31.263288  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 57380.7us
I20260709 13:02:31.276558 15009 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.277115 15332 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.286721  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:31.282693 15419 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:31.302717 15262 consensus_meta.cc:287] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.001s
I20260709 13:02:31.309693  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.314803  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.297993 15414 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:31.317137  8745 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:31.320870  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 106704us
I20260709 13:02:31.322301  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.316547 15332 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.300257 15250 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.316213 15280 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:31.327863  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:31.318725 15142 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.334854 15424 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
W20260709 13:02:31.335448 15386 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.458s	user 0.003s	sys 0.000s
I20260709 13:02:31.323993 15355 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:02:31.335853  8745 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:31.336716  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1134) took 1003 ms. Trace:
I20260709 13:02:31.337093  8815 rpcz_store.cc:276] 0709 13:02:30.332933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.333154 (+   221us) service_pool.cc:224] Handling call
0709 13:02:30.369709 (+ 36555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.370044 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.370074 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.389470 (+ 19396us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.732863 (+343393us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:30.733591 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.733693 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.334728 (+601035us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:31.336449 (+  1721us) raft_consensus.cc:1685] finished
0709 13:02:31.336470 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.336656 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2169,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5616,"spinlock_wait_cycles":8055552,"thread_start_us":302883,"threads_started":1,"wal-append.queue_time_us":343192}
I20260709 13:02:31.339876  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:31.327037 15418 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:31.317788 15151 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:02:31.320565 15262 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.312939 15217 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.331724 15414 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.321014 15419 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.331568 15280 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.357214 15423 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:31.330339 15396 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:31.286099 14923 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.383708 15322 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.384447 15357 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:31.385624 15406 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:31.330667 15108 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:31.318111 15009 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:02:31.354216 15424 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:31.390570  8742 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 21: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.349175 15397 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:31.364961 15380 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.397320 15266 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.397579 15380 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.376538 15418 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:31.386474  8890 raft_consensus.cc:2368] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22.
I20260709 13:02:31.399370 15418 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.312366 14821 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.401901 15395 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:31.403183 15225 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:31.403600 15225 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:31.404441 15225 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.390874 15423 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.396467 15228 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.397989 15400 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.399868 15380 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.382407 15271 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.401937 14821 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.395236 15217 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.444582 15322 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.445552 15262 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.386739  8817 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:31.416538 15397 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.448570 15397 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.417245 15419 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.348488 15407 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:02:31.405884 15414 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.418035 15116 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:31.419186  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.462356  8815 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:31.419909  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.420542  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.421396  8821 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:31.421824  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.467077  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.468628  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.421939  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.416803 15225 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:31.419188 15250 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:31.461570  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 102377us
I20260709 13:02:31.470261  8823 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:31.472800  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:31.473948  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.474540  8818 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.478590  8749 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:31.449954 15031 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.479207  8816 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:31.485877 15076 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.486094 15439 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:31.486469 15439 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.487393 15439 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.488725 15439 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:31.488988 15437 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:31.489526 15437 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:31.489917 15437 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.500736 15157 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:31.501417 15157 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.503504 15157 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.501431 15217 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:31.504669 15217 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.506726  8818 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:31.510016 15217 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.509928 15437 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.507428 15418 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.512861 15435 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:31.512564 15418 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.513932 15031 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.514271  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2601) took 2662 ms. Trace:
I20260709 13:02:31.514827  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1127) took 1422 ms. Trace:
I20260709 13:02:31.493363 15395 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:02:31.516676  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.514650  8740 rpcz_store.cc:276] 0709 13:02:28.852166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.024143 (+171977us) service_pool.cc:224] Handling call
0709 13:02:30.005239 (+981096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.005741 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.005768 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.022338 (+ 16570us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.419737 (+397399us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b180031e160. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:30.493846 (+ 74109us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:30.493967 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.494122 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.210580 (+716458us) raft_consensus.cc:1685] finished
0709 13:02:31.210611 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.514089 (+303478us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540016e4d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:31.514193 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4063,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":7992,"spinlock_wait_cycles":2265336832,"thread_start_us":73573,"threads_started":1,"wal-append.queue_time_us":573594}
I20260709 13:02:31.516928 15437 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:31.515313  8820 rpcz_store.cc:276] 0709 13:02:30.092493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.093395 (+   902us) service_pool.cc:224] Handling call
0709 13:02:30.111848 (+ 18453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:30.116519 (+  4671us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:31.223039 (+1106520us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b18001cace0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:02:31.322338 (+ 99299us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.392569 (+ 70231us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:31.405611 (+ 13042us) log.cc:844] Serialized 52 byte log entry
0709 13:02:31.405755 (+   144us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:31.405789 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.405871 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.514565 (+108694us) raft_consensus.cc:1685] finished
0709 13:02:31.514641 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.514774 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":90544,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1627,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":3901,"spinlock_wait_cycles":4141312,"thread_start_us":144624,"threads_started":2,"wal-append.queue_time_us":86567}
I20260709 13:02:31.521564  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.522140  8743 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.522883 15437 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.524259 15418 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.486474 15076 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.526010 15346 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.527139 15076 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:31.494910 15423 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.528342  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.523358 15437 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.530479 15332 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.532538  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.532769 15332 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.533192  8815 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:31.540709  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.542483  8788 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.544356  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.534221 15399 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:31.545258 15418 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:31.546686 15418 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.547335 15418 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:31.548867 15376 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:31.549342  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1120) took 1013 ms. Trace:
I20260709 13:02:31.549721  9440 rpcz_store.cc:276] 0709 13:02:30.535722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.556241 (+ 20519us) service_pool.cc:224] Handling call
0709 13:02:31.549128 (+992887us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400660154. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:31.549265 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679614208}
I20260709 13:02:31.553884  8889 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:31.558987  8743 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:31.553114  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.565734  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.566061  8896 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:31.570376 14821 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.574726 14821 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.576972 14821 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.531445 15437 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:31.583037 15147 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:31.583814 15001 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:31.584400 15001 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.586647 15001 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.583613 15147 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.587284 15419 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:31.587922 15419 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.589773  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.590741  8743 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.588512 15147 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.593591  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.594332  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.594348  9440 raft_consensus.cc:2393] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.595430  8896 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:31.596149  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.596370  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.596913  9448 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:31.596930  9441 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:31.599594  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.600346  8897 raft_consensus.cc:2393] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:31.600744  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:31.601181  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:31.602260  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:31.602734  8740 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:31.602950  8897 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:31.601307  8891 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:31.604719 15054 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.611331  8891 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:31.614100  8896 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:31.622189  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.640322  9447 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:31.592088 14821 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.602499 15419 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.646723  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.647918  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:31.647971  9448 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:31.650168  9447 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.651443  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:31.651643  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.656956  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.664964  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.665791  8891 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:31.666980  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:31.668927  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.669786  8896 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:31.675004 15403 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:31.690770  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.699359  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.701220  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.701740  8822 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:31.702829  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:31.704413  8745 raft_consensus.cc:1275] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:31.700117  8816 raft_consensus.cc:2393] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:31.704962  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.702870  8740 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.707922  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.700052  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:31.670678 15001 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:31.708684  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.674098 15147 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.716621  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:31.715183  8743 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:31.689544 15134 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:31.719022  8746 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.721483  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.646560 14821 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.724776  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.724079  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.697882 15054 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.737522  8816 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:31.738958  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:31.706797 15332 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.743144  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.744167  8815 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:31.700855 14923 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.750002  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.700306 15390 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:02:31.700659 15284 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:02:31.735553 15304 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:31.751907  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:31.723655 15217 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:31.754565  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.755373  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.756716  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.762012  8744 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.762212  8749 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:31.763077  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.763434  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:31.763597  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55993.7us
I20260709 13:02:31.740309 15031 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:31.764312 15031 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:31.764611  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.763778  8744 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:31.764741 15031 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.758138  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.712819 15145 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:02:31.764822 15457 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:31.753333  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.773584  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.774789  8741 raft_consensus.cc:2368] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:02:31.706874 15250 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:02:31.777014  8740 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:31.778779  8740 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:02:31.781733  8745 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:31.787434  8818 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:31.787523  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.750000 15054 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.792068  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.793138  8743 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:31.789345  8818 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:31.779922 15457 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
W20260709 13:02:31.814836  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 48810.1us
I20260709 13:02:31.815737  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:31.818161  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 118239us
I20260709 13:02:31.759599 15332 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.760767 14923 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.832930  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.835798  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.814265 15457 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.890633  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.892725  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.908321  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.909983  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.911415  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.915342  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.916435  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.917610  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:31.918553  8744 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:31.920310  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:31.831175 15290 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.911823 15401 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:31.925172 15447 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:02:31.925601 15225 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.946607 15390 consensus_queue.cc:1048] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:31.947050 15225 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:31.949105  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2895) took 1180 ms. Trace:
I20260709 13:02:31.926044 14821 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.927035 15054 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:31.929340  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.910781 15464 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.962172 15338 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:31.962491 15439 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:31.962354 15232 consensus_queue.cc:1175] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b60591217ee24378bb66a86d6d0dda52" responder_term: 17 status { last_received { term: 17 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:31.940408  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:31.965595  8891 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:31.949555  8893 rpcz_store.cc:276] 0709 13:02:30.768650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.768801 (+   151us) service_pool.cc:224] Handling call
0709 13:02:30.789100 (+ 20299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.789432 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.789461 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.806801 (+ 17340us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.948551 (+1141750us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b18000ac7e0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1
0709 13:02:31.948706 (+   155us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:31.948739 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.948811 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.948850 (+    39us) raft_consensus.cc:1685] finished
0709 13:02:31.948861 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.949049 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47648,"raft.queue_time_us":135137,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":6751,"spinlock_wait_cycles":11134336,"thread_start_us":69753,"threads_started":1,"wal-append.queue_time_us":452163}
I20260709 13:02:31.966451 15440 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:31.966976 15440 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:31.968065 15440 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:31.959654 15390 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:02:31.973513 15381 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.371s	user 0.003s	sys 0.003s
I20260709 13:02:31.974903 15332 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:31.910332 15009 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.961226  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.960054 15290 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.964277  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:31.992624  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 289891us
I20260709 13:02:31.996632  9414 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:32.018671  8818 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:02:32.042790  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.042606  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46099.6us
I20260709 13:02:32.043769  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:32.056555  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:32.057204  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 93020.3us
I20260709 13:02:32.060603  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:31.982962 15411 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:32.063781  8818 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:31.981377 15440 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:32.072785  8818 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:32.055123 15402 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.065640  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.057231 15369 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:32.055467 15250 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:32.083739  8818 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:32.058305 15380 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.067473  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 251860us
I20260709 13:02:32.069766 15461 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.017710 15009 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.070468 15115 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:32.101308 15134 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:32.102629 15134 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.070396 15225 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.073137 15437 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.114450 15437 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:32.114761  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.086939 15402 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.115370  8743 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:32.116289  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.117089 15402 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.092738 15369 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:32.119431  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 2983) took 1019 ms. Trace:
I20260709 13:02:32.119843  8817 rpcz_store.cc:276] 0709 13:02:31.099634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.099752 (+   118us) service_pool.cc:224] Handling call
0709 13:02:32.119230 (+1019478us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:32.119374 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591165696}
I20260709 13:02:32.120893  9414 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:32.092358 15414 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:32.110271 14923 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.104604 15250 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:32.090350 14821 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:02:32.123359 15439 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:32.166047 15439 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:32.167127 15439 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.170986 15421 raft_consensus.cc:2764] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:02:32.173216 15235 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:32.173581  9442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44010 (request call id 1123) took 1637 ms. Trace:
I20260709 13:02:32.174247  9442 rpcz_store.cc:276] 0709 13:02:30.536575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.538204 (+  1629us) service_pool.cc:224] Handling call
0709 13:02:32.173359 (+1635155us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54006fe0d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:32.173512 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522382720}
I20260709 13:02:32.099604 15380 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.178282 15380 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.098866 15376 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:32.133720 15145 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:02:32.180577 15145 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:32.162052 15271 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.164770 15142 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.118287 15369 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.183923  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:32.135758 15455 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 13:02:32.182180  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58813.2us
I20260709 13:02:32.189622  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:32.118829 15115 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.173676 15235 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:32.165516 14923 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:32.137677 15054 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:32.194257  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.201104  8741 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
W20260709 13:02:32.195217  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 106310us
I20260709 13:02:32.197976  8818 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:32.203174  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.203744  8818 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:32.181953 15271 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.123584  8743 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:32.199110  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.194859 15466 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.220381 15054 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.195326 15138 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:32.235239  9442 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:32.220158  8863 leader_election.cc:400] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:02:32.244902  8863 leader_election.cc:403] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:32.194481 15380 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:02:32.200768 14923 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.192934 15142 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.246353  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:32.248147  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45032.7us
I20260709 13:02:32.240305 15147 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:32.246063 15138 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.261906  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.252861 15376 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.252748 14923 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:32.248579 15145 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:02:32.243450 15466 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.274356 15138 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:32.294703  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.298691 15481 log.cc:927] Time spent T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:02:32.302294 15147 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:32.302696 15478 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:02:32.302881 15147 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:32.307545 15473 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:32.275964 15411 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.244218 15250 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.293406 15414 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:32.259434 15235 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.281378 15031 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.319793 15338 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.329139 15031 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:32.289238 15486 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:32.324368 15009 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.328259 15235 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.330842 15486 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:32.331771 15486 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.337831 15466 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:32.343034 15147 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.343631  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:32.344092  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.319059 15457 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.344759  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:32.346128  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.346674 15457 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.347234  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.348157  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:32.349045  8740 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:32.353075  9446 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:32.322152 15479 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:02:32.354511 15142 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.355189  9442 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:32.324888 15411 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.354465  9443 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:32.359952  8745 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:32.362129  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.362550  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1144) took 2098 ms. Trace:
W20260709 13:02:32.365101  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.366020  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.366044  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.364282  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.367565  9449 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:32.363010  8895 rpcz_store.cc:276] 0709 13:02:30.264393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.264528 (+   135us) service_pool.cc:224] Handling call
0709 13:02:30.264885 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.307751 (+ 42866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.307801 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.308262 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.897187 (+1588925us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b180028dc80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:32.070260 (+173073us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:32.070312 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.076478 (+  6166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.358201 (+281723us) raft_consensus.cc:1685] finished
0709 13:02:32.358223 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.362486 (+  4263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32088,"raft.queue_time_us":59204,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":8393,"spinlock_wait_cycles":3224448,"thread_start_us":172864,"threads_started":1,"wal-append.queue_time_us":1855936}
I20260709 13:02:32.368378 15116 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:02:32.371078  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.371558  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.373617  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.362388 15051 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.375422 15051 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.375712  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.376386  9449 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:32.377137 15266 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:32.370781 15380 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:32.377610 15266 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:32.378015 15380 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:32.377700 15051 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.378058 15228 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:32.378744 15266 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.379889  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.356922 15271 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.378988 15380 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.382920  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:32.395510  8895 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.384268  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.397048  8898 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:32.385484  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.362619 15440 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:32.391638  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:32.404485  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.405687  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:32.406855  8897 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:32.409359  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:32.409194  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:32.411048  9443 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:02:32.413619  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:32.411098  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 136089us
I20260709 13:02:32.415926  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.367426 15421 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.422816  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:32.329164 15338 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:32.434574  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.427619  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 166086us
I20260709 13:02:32.428804  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:32.436729  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.394498 15409 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.394207 15427 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:32.438597  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.450635  8745 raft_consensus.cc:2410] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:02:32.420334 15051 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:32.428929  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.441984  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.380232 15487 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:32.455250  8897 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:32.426082 15116 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.458469  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.452927  8821 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:32.430984 15271 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.461692 15271 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:32.461470 15487 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.461131 15116 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.462725 15116 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.464416 15487 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:32.465538  8897 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:32.462188 15271 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:32.472793 15476 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:02:32.476972 15054 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.478328 15492 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:32.478864 15492 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.480821 15492 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.393743 15341 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
W20260709 13:02:32.482246 15449 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.735s	user 0.001s	sys 0.000s
I20260709 13:02:32.403810 15228 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:02:32.347267 15138 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.483690  8749 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.444823 15409 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.421128 15440 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.491114 15461 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.432188 15421 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.458106 15051 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.506755  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.510403  8892 raft_consensus.cc:2410] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:02:32.510321  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 108436us
I20260709 13:02:32.507757  8749 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:32.512686  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.484197  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:32.476779 14923 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.511070  8862 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:32.503204 15493 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:32.515317  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.518172  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:32.520493  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.521462 15428 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.522022 15428 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:32.522423 15428 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.528389  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.534053  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.535269  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.535857  8817 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:32.536824  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.530210  8862 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.541440  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:32.546499  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.546797  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.547931  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.514379 15051 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:32.558235  9447 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:32.556998 15225 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:32.578543 15374 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:32.583976 15374 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.586213 15374 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.579664 15116 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.588166  8817 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:32.588238 15116 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.589650 15280 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.592710  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:32.593343  8747 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.603052  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:32.603816  8818 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:32.604126  8747 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
W20260709 13:02:32.516382 15392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.136s	user 0.026s	sys 0.017s
W20260709 13:02:32.606025  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 75935.5us
I20260709 13:02:32.606997  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.612090 15428 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.518723 15414 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:32.545379 15290 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.005s
I20260709 13:02:32.605417  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:32.624974  8894 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:32.617569  8895 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:32.620718  8818 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:32.616499  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.558740 15054 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.628839  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:32.615585 15392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.026s	sys 0.018s
I20260709 13:02:32.629369  8820 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:32.630642  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2680) took 2271 ms. Trace:
W20260709 13:02:32.629266  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 192254us
I20260709 13:02:32.630784  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.633761  8890 rpcz_store.cc:276] 0709 13:02:30.358797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.505852 (+147055us) service_pool.cc:224] Handling call
0709 13:02:32.624554 (+2118702us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b540046df54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:32.630584 (+  6030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021393664}
I20260709 13:02:32.634143  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.632710  8895 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:32.627182  9448 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:32.636394  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.638536  8818 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:32.638773  8815 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:32.635939 15390 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.635932 15228 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:32.630426 15054 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:32.648062  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:32.649489  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.653326  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:32.640763 15494 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:32.657366  8895 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:32.713125  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:32.714082  8895 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.718777  8816 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:32.644189 15411 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.657171 15054 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:32.725848  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 76483us
I20260709 13:02:32.710008 15500 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:32.716065 15289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.004s	sys 0.007s
I20260709 13:02:32.710829 15228 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:32.648442 15142 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:02:32.710292 15392 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:02:32.745507 15490 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.214s	user 0.002s	sys 0.000s
I20260709 13:02:32.750991 15051 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.752068 15434 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:32.754094 15502 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:32.757391 15496 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:32.757915 15505 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:32.656447 15271 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.760449 15440 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:32.722251 15338 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.655007 15414 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:32.746412 15411 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:32.654170 15487 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 13:02:32.735644 15376 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17.
I20260709 13:02:32.831246  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 1154) took 1287 ms. Trace:
I20260709 13:02:32.831724  8889 rpcz_store.cc:276] 0709 13:02:31.543875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.544074 (+   199us) service_pool.cc:224] Handling call
0709 13:02:32.830554 (+1286480us) spinlock_profiling.cc:243] Waited 8.79 ms on lock 0x7b540045abd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:32.831181 (+   627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20213632}
W20260709 13:02:32.813476  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54041.3us
I20260709 13:02:32.822634  8895 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:32.758689 15409 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:32.770956 15289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.023s	user 0.005s	sys 0.008s
I20260709 13:02:32.656019 15390 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.821550  8890 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:32.731966 15332 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:02:32.809674 15428 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.718411 15494 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.730192 15506 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:32.869449  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.822603 15421 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.823391 15414 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:32.871212  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:32.882495 15477 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:32.882768  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.883273 15477 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.884166  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.885749 15503 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:02:32.886461 15414 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.888142 15051 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:32.889393  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.890635 15501 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:02:32.892935  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.867455 15496 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:32.894516  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.895149  8893 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:32.894866 15496 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.896173  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2422) took 1368 ms. Trace:
I20260709 13:02:32.896543  8893 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:32.897886  8893 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:32.898202  8893 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:32.875698 15228 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.892963 15054 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:32.901584  8889 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:32.885784  8815 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:32.902590  8893 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.903715  8893 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:32.872287 15289 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:32.907080 15289 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:32.885967 15439 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:32.896582  8820 rpcz_store.cc:276] 0709 13:02:31.527810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.528073 (+   263us) service_pool.cc:224] Handling call
0709 13:02:32.895991 (+1367918us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000811d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:32.896121 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531090304}
W20260709 13:02:32.921902  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.824375 15440 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:02:32.929071  8898 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:32.929237  8819 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:32.830586 15411 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:32.930903 15508 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:32.931669  8893 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:02:32.932040  8893 pending_rounds.cc:85] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 7
I20260709 13:02:32.932260  8893 pending_rounds.cc:107] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:02:32.932482  8893 raft_consensus.cc:2889] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:32.933562  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:32.934224  8893 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:32.934943  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:32.936482  8895 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:32.947793  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:32.950335  8895 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:32.923209 15054 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:32.952474  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:32.978159  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.957173 15491 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.610s	user 0.004s	sys 0.000s
I20260709 13:02:32.981436  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:32.990182  8820 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:32.985874 15477 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:32.987789 15510 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:02:33.001871 15511 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:33.003499 15494 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:32.956825 14923 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.017604 15414 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.051620 15414 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.036433 15271 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.032174 15494 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:33.052874 15494 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.053161  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:33.055707 15494 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.058000  8822 raft_consensus.cc:1217] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Deduplicated request from leader. Original: 14.7->[15.8-15.8]   Dedup: 15.8->[]
I20260709 13:02:33.060070  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1141) took 1970 ms. Trace:
I20260709 13:02:33.063621  8822 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:33.060730  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:33.066918  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:33.068517  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.063752  8824 rpcz_store.cc:276] 0709 13:02:31.089522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.089647 (+   125us) service_pool.cc:224] Handling call
0709 13:02:31.089982 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.167593 (+ 77611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.167639 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.168261 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.283618 (+115357us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:31.283661 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.057258 (+1773597us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b5400076a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:33.057332 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.057375 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:33.057387 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.060003 (+  2616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12438,"raft.queue_time_us":10711,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":10662,"spinlock_wait_cycles":157914880,"thread_start_us":58654,"threads_started":1,"wal-append.queue_time_us":125357,"wal-append.run_cpu_time_us":2041,"wal-append.run_wall_time_us":1080521}
I20260709 13:02:33.074704  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.075229  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.080251  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:33.081396  8824 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:33.068023 14923 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:33.095650  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:33.103847  8744 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:33.104159  8822 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:33.064103 15250 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:02:33.073546 15332 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.110263  8747 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:33.106868 15519 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:02:33.118718  8891 raft_consensus.cc:2410] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:02:33.123065 15416 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:02:33.125794 15332 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.127262  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2369) took 1366 ms. Trace:
I20260709 13:02:33.127650  8749 rpcz_store.cc:276] 0709 13:02:31.761110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.764392 (+  3282us) service_pool.cc:224] Handling call
0709 13:02:33.127084 (+1362692us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54004637d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:33.127203 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652614016}
I20260709 13:02:33.128144 15332 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.103649 14923 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.130745 14923 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.108912 15001 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.132395 15001 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.122802 15502 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:33.133994 15502 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:33.134358 15502 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:33.134647 15001 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.135167 15502 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.136405 15502 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:33.146065  8749 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:33.157832  8749 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:33.163863  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:33.165004  8818 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:33.190757  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.195616  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.214222  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.214749  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.214879  8889 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
W20260709 13:02:33.221138  8742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45274 (request call id 1146) took 2526 ms (client timeout 1893 ms). Trace:
I20260709 13:02:33.221230  8824 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.221402  8741 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:33.221726  8742 rpcz_store.cc:269] 0709 13:02:30.694603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.696792 (+  2189us) service_pool.cc:224] Handling call
0709 13:02:33.220927 (+2524135us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54004632d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:33.221080 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553466752}
W20260709 13:02:33.229176 15445 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.478s	user 0.001s	sys 0.004s
I20260709 13:02:33.230757 15138 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.231352 15440 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.232080 15440 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.235013 15440 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:33.242384 15458 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.478s	user 0.006s	sys 0.000s
I20260709 13:02:33.245764  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:33.246598  8895 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.205868 15250 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:33.253080 15250 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.255947 15250 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.259166  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:33.261727  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50899.1us
I20260709 13:02:33.147385 15414 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.268307  8742 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.147729 15400 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.221506  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.194854 15439 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.272356  8818 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:33.282581  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.283008  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.283857  8817 raft_consensus.cc:2410] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 20 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 15.
I20260709 13:02:33.272374  8744 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:33.273794  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.210899 15477 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.274840  8712 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:33.290897  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:33.292438 15051 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:33.292995 15051 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:33.293501 15051 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.297585  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.300321 15115 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.270887 15376 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.305255  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.314533  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.314630  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:33.316143  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:33.317734  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.322118  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.326154  8895 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:33.326510  8818 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:33.328583  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:33.329563  8744 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:33.333209  8818 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.336190  8747 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:33.302891 15376 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.341228  8747 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.330951 15304 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:33.348332 15051 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.273064 15414 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.320065 15228 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.282305 15400 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.319517 15503 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.289189 15439 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.319376 15054 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:33.310869 15477 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.366392  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:33.377346 14923 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.379232  8744 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:32.415750  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:33.357017 15304 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:33.390388  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.384586 15524 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:02:33.406170  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.422115  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.434901 14923 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.436427 15369 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.436616 15376 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.436982 15369 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.439262 15369 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.440946 15142 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:33.386863 15332 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:33.444983 15427 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:33.390470 15414 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.439761 15369 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.370234 15503 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.378396 15054 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.452942  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.453884  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.456467  8890 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:33.367667 15228 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.458931  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.465512  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.461709  8787 leader_election.cc:400] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22.
I20260709 13:02:33.463647  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.442523 15051 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.467000  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:33.447219  8644 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 13, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 13 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:02:33.475270  8787 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.476560  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:33.476845  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.482473  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.447922 15427 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:33.483784  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:33.486415  8889 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:33.487150  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.487685  8890 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:02:33.487761  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:33.488226  8890 pending_rounds.cc:85] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:02:33.488464  8890 pending_rounds.cc:107] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:33.486342  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.488704  8890 raft_consensus.cc:2889] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.489153  8895 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:33.490545  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.460496 15232 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:33.446935 15142 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.462728 15369 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.486132 15418 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:33.498279  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.476369 15304 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.516093 15293 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:33.561169  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 63022.5us
I20260709 13:02:33.499464 15289 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:33.516952 15414 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:33.497015 15332 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:33.486289 15496 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:33.541359 15390 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.568310  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.485658 15051 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:02:33.519037 15232 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.565095 15439 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.571090 15492 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.586390 15496 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:33.586787 15293 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:33.587664  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.607117 15293 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:33.607728 15293 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.608225 15293 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.608385  8787 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.544976 15427 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.610177 15427 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.613682 15418 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:33.616154 15418 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.618465 15280 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:33.626477  8644 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:33.630520 15503 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.632296 15138 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.633471 15228 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.632890 15138 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.635735 15369 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.636128 15138 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.629508  8644 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:33.637349  8787 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.638608 15322 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:33.640090 15518 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.640725 15518 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.644630  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:33.646606 15138 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.649657  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2673) took 3293 ms. Trace:
I20260709 13:02:33.649742 15322 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:33.650386 15322 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.650982 15322 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:33.649632 15138 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.653036 15138 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.653329 15411 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.650138  8894 rpcz_store.cc:276] 0709 13:02:30.356623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.557949 (+201326us) service_pool.cc:224] Handling call
0709 13:02:32.624723 (+2066774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.625619 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.649473 (+1023854us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540046df50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:33.649600 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4753539456}
I20260709 13:02:33.654542  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:33.654947 15411 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:33.655319 15289 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:33.655757 15289 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.656106 15289 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:33.658656 15529 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.282s	user 0.003s	sys 0.000s
W20260709 13:02:33.660763 15530 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:02:33.662348  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1159) took 1211 ms. Trace:
I20260709 13:02:33.662748  8823 rpcz_store.cc:276] 0709 13:02:32.450636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.040397 (+589761us) service_pool.cc:224] Handling call
0709 13:02:33.040746 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.041167 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.041208 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.041777 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.141146 (+ 99369us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:33.141172 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.141219 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.660115 (+518896us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:33.661995 (+  1880us) raft_consensus.cc:1685] finished
0709 13:02:33.662024 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.662277 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10250,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":104,"spinlock_wait_cycles":5567744,"thread_start_us":71907,"threads_started":1,"wal-append.queue_time_us":314586}
I20260709 13:02:33.665544  8891 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:33.669848 14923 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.670372 14923 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.671545 15289 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.574031 15400 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:33.591156 15489 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 1.240s	user 0.004s	sys 0.009s
I20260709 13:02:33.672478 14923 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.581506 15419 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.675372 15232 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.678894  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:33.680361 15322 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.683537 15289 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.690778  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:33.691577  8897 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.692822  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:33.701977  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.570591 15414 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.711773  8894 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:33.716534  8741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1166) took 1620 ms. Trace:
I20260709 13:02:33.580821 15477 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:33.718896  8741 rpcz_store.cc:276] 0709 13:02:32.095895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.222635 (+126740us) service_pool.cc:224] Handling call
0709 13:02:33.221055 (+998420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.221898 (+   843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.716282 (+494384us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54004632d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:33.716461 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2255190784}
I20260709 13:02:33.719120  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.728598  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.724997  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.730090  8824 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:33.724725 15400 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:33.735472 15400 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.736145 15401 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:33.736457 15492 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:33.637465 15054 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:33.748400  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.755208  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 383916us
I20260709 13:02:33.752054  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:33.706233 15503 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:33.572914 15332 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.731567 15289 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:02:33.762924  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 2992) took 2025 ms. Trace:
I20260709 13:02:33.764917  8819 rpcz_store.cc:276] 0709 13:02:31.737277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.762612 (+ 25335us) service_pool.cc:224] Handling call
0709 13:02:32.897040 (+1134428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.929853 (+ 32813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.445107 (+515254us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b54000811d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:33.762727 (+317620us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000811d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:33.762862 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682051712}
I20260709 13:02:33.751698 15419 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.758694 15280 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:33.874199  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.852270  8897 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
W20260709 13:02:33.882898 15520 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:33.755509 15477 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:02:33.875751 15289 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:33.870242 15400 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.962903  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:33.938598 15503 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:33.955365 15532 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:33.942792 15266 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.869447 15419 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:33.950042 15142 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.977391 15503 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.916512 15280 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.986552 15142 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:33.987102 15142 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.990629 15400 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.991243 15400 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:33.969102 15322 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:33.996608 15428 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:33.917150 15496 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:34.013516  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.072197  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 220255us
I20260709 13:02:33.994360 15419 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:34.072225 15115 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.079120 15542 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:02:34.100389 15115 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.102797 15115 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:33.996683 15293 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:34.103462  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 364722us
W20260709 13:02:34.107009  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 381994us
I20260709 13:02:34.086265 15546 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.111265  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.113824  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:34.093080 15477 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.098114 15540 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.099920 15427 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.119647  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.121886  9445 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:34.127195  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.130404  9442 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:34.130931  9444 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:34.113618 15546 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.139855 15546 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.140574 15546 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.116434 15332 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.109084 15414 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:34.153160  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.153527  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:34.156198  9440 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:02:34.157876  9449 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:34.141099 15546 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.147900 15142 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:34.160215 15400 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.165601 15293 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:34.123690 15540 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.170733 15540 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:34.171558 15280 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.123837 15477 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.191172  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:34.192250  9440 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.206182  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.208189  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.209151  9440 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:34.212386  9449 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.213267  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:34.222249  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 259603us
I20260709 13:02:34.227547  8893 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:34.207568 15554 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:34.237779  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2900) took 1851 ms. Trace:
I20260709 13:02:34.238433  8896 rpcz_store.cc:276] 0709 13:02:32.386113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.386887 (+   774us) service_pool.cc:224] Handling call
0709 13:02:32.415425 (+ 28538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.626752 (+1211327us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b180040bee0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f64c7060 00007f90f64c7bfb 00007f90f64d264b
0709 13:02:33.626966 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.627004 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.627645 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.995801 (+368156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:33.996007 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.996280 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.226986 (+230706us) raft_consensus.cc:1685] finished
0709 13:02:34.227023 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.237684 (+ 10661us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88525,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":3480,"spinlock_wait_cycles":2898816,"thread_start_us":465237,"threads_started":2,"wal-append.queue_time_us":548406}
I20260709 13:02:34.245002  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:34.245631  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.246150  9441 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.246371  9445 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:34.247153  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:34.247570  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.248517  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.250033  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.251804  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.253314 15293 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:34.254850  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.256290 15293 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:34.256808 15293 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.257165 15293 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.249223  9445 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:34.267721 15322 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:34.270339  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.272162 15419 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:34.272785 15419 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:34.274832 15419 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.238977  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:34.237818 15554 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.297166 15280 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.305090  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66231.4us
W20260709 13:02:34.303756  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 194370us
W20260709 13:02:34.310194  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.301304 15554 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.314294  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.311285 15054 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:34.318976 15560 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:02:34.321919 15293 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.324918 15293 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.331698 15560 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.332849  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:34.339154 15551 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:34.344132  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.344326  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.344802 15332 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:34.345939  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:34.346078  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:34.346546  8890 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:34.346911  8896 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:34.346724  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.347355 15477 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:34.348954  8895 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.351796  8897 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:34.350274  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.352731  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.353344  8894 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:34.354935  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.356389  8823 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:34.361521  8895 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:34.363060  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.363730  8894 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:34.357787  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.366895  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.367326  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.311348 15280 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.367190  8824 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
W20260709 13:02:34.371250  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.377691  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:34.387734  8890 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:34.391664  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.391666  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.393263  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:34.396068  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62543.6us
I20260709 13:02:34.397517  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.398145  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.353618 15054 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:34.353511 15546 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:34.407068  8822 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.408376  8889 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:34.414145  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:34.420586  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.421444  8821 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:34.422358  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.390280 15414 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:02:34.433070 15400 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:34.434444  8748 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:34.436118  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.436466 15280 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.437362  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:34.437948  8740 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:34.438227  8742 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:34.438465  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.423209 15492 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.445637  8822 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:34.442754  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:34.443485  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.450192  8824 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:34.446632  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.451323  8815 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:34.451790  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.452628  8815 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.454557 15054 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.456607  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.457691 15496 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:34.458438  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.458976  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.458545  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.459729  8744 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:34.460639  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.459828  8746 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:34.461304  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.463188  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.463286  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.464845 15576 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.464811  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.465698 15576 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.467376  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.470193  8746 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:02:34.471480  8712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.055s	sys 0.063s
W20260709 13:02:34.471801  8712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.139s	user 0.055s	sys 0.063s
I20260709 13:02:34.473172  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.473948  8744 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.475452  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.477123  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.477792  8744 raft_consensus.cc:2393] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.477927  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.479132  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.480985  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.481843  8744 raft_consensus.cc:2410] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 22 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:02:34.482890 15496 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:34.472293  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 1.18163e+06us
I20260709 13:02:34.485807  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.486112  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.486913  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:34.494585  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.496802  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.499651  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.505642 15560 raft_consensus.cc:2764] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:02:34.506083 15582 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:34.506739 15576 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:34.514142  8715 leader_election.cc:400] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:34.514492  8715 leader_election.cc:403] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:02:34.507247 15576 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.517874  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.521557  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.515167 15560 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.453816 15492 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.489491 15546 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.527086  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.462759 15280 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
W20260709 13:02:34.532392  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.507592 15559 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.534735  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.533481  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.524219 15563 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.548863  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.550884  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:34.551615  9449 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:34.538547  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.525754 15419 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:34.550269 15560 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:02:34.555071 15115 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.554860 15554 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.555563 15115 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.557420 15346 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:34.557950 15346 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:34.558846 15346 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.546445 15054 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.565470  9441 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:34.515774 15380 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:02:34.568627  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.557066 15554 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:34.519927 15293 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:02:34.575594  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:34.575672  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 91616.5us
I20260709 13:02:34.539816 15557 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:34.577270  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.525548 15322 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:02:34.581954 15318 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:34.582908 15573 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:02:34.584187  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.585021 15411 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:34.585485 15115 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.571025 15546 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.550360 15540 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:34.572320 15581 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.601071  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 147493us
I20260709 13:02:34.579283 15400 consensus_queue.cc:1243] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:34.599594 15318 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:34.617641  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.619591  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.588102 15054 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:34.621829  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:34.625480  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.599292 15266 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:34.609902 15411 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:34.635946  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:34.638254  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 88904.4us
I20260709 13:02:34.631037 15554 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:34.646443  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.631326 15477 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.647962  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:34.651297  8822 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:34.647259  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57702.1us
I20260709 13:02:34.639634 15496 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.639030 15400 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:34.614540 15584 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:34.639242 15558 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:34.637178 15054 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:34.646839 15380 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.617193 15581 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.640362 15115 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:02:34.686885  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.699748  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:34.677505  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:34.668867 15411 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.675696 15280 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.641844 15266 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.702250 15411 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:34.709693 15054 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.657925 15318 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.663018 15586 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.696880 15581 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:34.725939  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 77534.1us
W20260709 13:02:34.675148  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 91027us
I20260709 13:02:34.746079 15318 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.657341 15477 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:34.699565 15400 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.725994 15518 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:34.731993  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 95666.6us
I20260709 13:02:34.775983  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:34.729184 15586 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.782599 15586 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:34.737219 15599 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:34.757510 15576 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:34.763494  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 124359us
W20260709 13:02:34.770711  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 124368us
I20260709 13:02:34.775030  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.778461 15601 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:34.780511 15414 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:34.794265  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.802162  8712 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:34.802433  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.803102  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.676067 15592 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:02:34.806557  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.799748  9412 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:34.813630  8824 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 13.
I20260709 13:02:34.756026  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:34.811913  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.817327  8818 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:02:34.817965  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.814275  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:34.818657  8824 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
W20260709 13:02:34.824625  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 234836us
I20260709 13:02:34.815653  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:34.800735 15346 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.817755  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:34.808400 15597 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:34.832617  9412 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:34.828657  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:34.833235  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.809695 15559 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:34.841890  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.833227  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.809226 15414 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:34.839561  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:34.817056 15546 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:34.818209 15400 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:34.821020 15554 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.816241 15598 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:34.808027 15591 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.858683  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:34.860256  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:34.833369 15586 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.840386 15477 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.844376 15584 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:34.890383  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:34.891445  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:34.892212  8895 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:34.893481  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:34.807261 15599 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:34.923764  8897 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.862609 15518 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.856658 15115 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:34.844468 15559 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.924914  8896 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:02:34.882509 15142 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:34.928328  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:34.827812 15558 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:02:34.862888 15400 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:34.883325 15280 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.923197  8890 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.897302 15477 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:34.944443  8897 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:34.921435 15584 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.887307 15554 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:34.883234 15602 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:34.947878 15586 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.839301 15346 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.922699  8894 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:34.899880 15585 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:34.864579 15357 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:34.937866 15115 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:34.962688  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 145059us
W20260709 13:02:34.956943  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 155620us
I20260709 13:02:34.957957 15400 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:34.966573 15581 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.966157 15293 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:34.900633 15266 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:34.920954 15591 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:34.946380  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.991539  8890 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:34.965891 15477 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.939026 15440 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.966471 15142 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:34.949119 15559 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.974642 15054 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:35.000356 15554 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.995370 15546 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:35.000250 15602 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.896603 15598 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.082604 15602 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.084493 15598 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:35.037426  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.012673 15585 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.085968 15546 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.013490 15584 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:34.991138 15280 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:34.948038 15414 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.095554 15591 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.098441 15346 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:35.101925 15609 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:02:35.105826 15054 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:35.109726 15607 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 13:02:35.111590 15603 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:02:35.098428 15142 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.078305 15599 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.039784 15293 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.123842 15293 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.124634 15620 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.129220 15608 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.015029 15586 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.038416 15581 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.086675 15585 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.017067 15115 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:35.133050  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 300587us
W20260709 13:02:35.132220  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.129887 15608 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.129223 15618 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.151069  8862 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:35.148391  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:35.174096  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.176669  8864 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:35.144060 15293 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:35.178071  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.558s (SENT) [suppressed 9 similar messages]
I20260709 13:02:35.166363 15599 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.186998  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 326880us
I20260709 13:02:35.196830  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.200282  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.201251  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.170001 15518 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:35.203332  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:35.205231  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 255415us
I20260709 13:02:35.132045 15620 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.204010  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.203060  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.166570 15558 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:02:35.205269 15563 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.000s
W20260709 13:02:35.242848  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64808us
I20260709 13:02:35.243024 15518 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:35.243610 15518 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.244031 15518 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:35.240973  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.211002 15346 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:35.245229  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 433878us
I20260709 13:02:35.254923  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:35.196672 15623 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:35.241793  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.245149 15228 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:35.210209 15604 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:35.241446 15563 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:35.258224  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.260998  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.216112 15598 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:02:35.244735  9414 leader_election.cc:336] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.893s (SENT)
I20260709 13:02:35.168835 15618 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.271298  8896 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:35.195415 15293 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.273988  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.278949  8896 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:35.272783  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.254422 15346 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.293792  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.297106 15440 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:35.210781 15599 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.264562 15228 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:35.306031  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 102104us
I20260709 13:02:35.306840  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.306921  8744 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:35.307754  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.308557  8896 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:35.311856  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:35.312592  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:35.312752  8896 raft_consensus.cc:2368] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
W20260709 13:02:35.295675 15318 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.002s
I20260709 13:02:35.298588 15346 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.322809  8823 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:35.291621 15604 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:35.340936  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.308389 15632 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:35.345655 15624 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:02:35.351075 15572 log.cc:927] Time spent T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.878s	user 0.000s	sys 0.002s
I20260709 13:02:35.352433 15629 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:35.359805  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:35.360880  8894 raft_consensus.cc:2410] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:35.370147  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.371407  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.331334 15622 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:35.352885 15629 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:35.375043 15629 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.375459 15618 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.375988 15618 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:35.376451 15618 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:35.379931  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50264.9us
I20260709 13:02:35.379854 15268 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:35.381669 15142 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:02:35.382649 15620 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.327561 15318 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:35.395877  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.402289  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.353350 15581 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.353986 15632 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:35.381682 15586 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:35.388129 15142 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:02:35.415876 15618 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.383553 15228 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.418045 15228 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.396237 15318 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.423195 15318 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:35.425984 15610 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:02:35.428855  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.429499  8890 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:35.430299  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.431301  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.436251  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:35.439788 15503 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.443033 15115 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.443697 15280 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.449973 15293 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.432129 15604 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.453986 15477 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.454972 15584 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.456172 15477 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.458081  9443 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.462781 15599 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.409319 15632 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:35.414482 15608 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.471592  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1137) took 1363 ms. Trace:
I20260709 13:02:35.472018  9447 rpcz_store.cc:276] 0709 13:02:34.110249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.110381 (+   132us) service_pool.cc:224] Handling call
0709 13:02:34.110655 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.191277 (+ 80622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.191322 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.191387 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:34.191432 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.471368 (+1279936us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b5400430b54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:35.471411 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.471538 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184886528}
I20260709 13:02:35.472903 15115 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:35.457050 15584 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.473528  8897 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:35.474326 15400 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.476930 15400 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.481941 15554 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.483922 15518 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.484802 15559 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.485302 15585 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.496502  8890 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:35.500268  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:35.501125  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.503862  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.565361  8746 raft_consensus.cc:2393] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:35.505446  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.511631  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:35.566435  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 59884.9us
W20260709 13:02:35.566807  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.515969  8740 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:35.516201  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:35.550086  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 254568us
I20260709 13:02:35.557819  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:35.547061 15357 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.020s	sys 0.018s
I20260709 13:02:35.560482  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.574980  9414 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:35.562161  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 249665us
I20260709 13:02:35.571784  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:35.576990  8818 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 14.
I20260709 13:02:35.547617 15598 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:35.579578  8815 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:35.578735  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:35.582417  8895 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:35.587157  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.588514 15546 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:35.589023 15546 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.589533 15585 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.590883 15642 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:35.576017 15228 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.574208 15357 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:35.554891 15346 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.580154 15639 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:35.599573  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:35.604641  9447 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:35.605976  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:35.600829  8712 leader_election.cc:400] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:35.608026  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.601020  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:35.607465  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:35.591833 15614 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:35.562377 15629 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.616917  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:35.615669  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.579725 15619 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:02:35.621208  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.621397  8818 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:35.627074  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.616182 15584 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:35.582372 15643 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:02:35.582618 15602 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:35.633644  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 324429us
I20260709 13:02:35.617444 15614 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:35.636265  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.616529 15051 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:35.642298  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.631898 15584 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.643756  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.644070  8892 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:02:35.645280  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44369.1us
I20260709 13:02:35.645987 15617 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:35.646322 15645 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:35.646456 15584 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.649845 15554 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.647563 15617 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.652679 15617 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.648182 15559 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.653367 15357 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.657608  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.637919 15614 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:35.639333 15650 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:35.642670 15647 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:35.639237 15598 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.671036 15585 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:35.677704 15554 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.683463 15506 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.693732 15602 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:35.686800 15647 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:35.703559 15647 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.709463 15629 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:35.701627 15518 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:35.724989 15588 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:35.725692 15588 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:35.726773 15588 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.728431 15588 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.713060 15584 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.715572 15647 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:35.839885 15647 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:35.809072 15632 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.843336 15632 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:35.846009 15654 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
I20260709 13:02:35.862152 15614 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:35.907734  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 262331us
W20260709 13:02:35.908986  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.912956  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 269178us
I20260709 13:02:35.914729  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.938203  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.939810  8740 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:35.943079  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.947386  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:35.948084  8740 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:35.948410  8747 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:35.953580  8740 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:35.965140  8740 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:35.967718  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:35.976207  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 318595us
I20260709 13:02:35.976511  8749 raft_consensus.cc:2410] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:35.970588  8747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:35.977676  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:35.977813  8747 raft_consensus.cc:606] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:35.978469  8747 raft_consensus.cc:614] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:35.980417  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.007982  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.014709  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.016017  8749 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:36.018983  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.020498  8749 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:35.809072 15584 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.049068 15115 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:36.059156 15440 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:36.064700 15557 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:36.051800 15280 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:36.074774 15630 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.075832 15630 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.078583 15346 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.079658 15639 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:36.079883  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:35.698100 15506 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.086536  9441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1214) took 1841 ms. Trace:
I20260709 13:02:36.093534  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.093269  9441 rpcz_store.cc:276] 0709 13:02:34.244610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.251063 (+  6453us) service_pool.cc:224] Handling call
0709 13:02:34.251413 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.568008 (+316595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.085836 (+1517828us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540067c354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.086275 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721956224}
I20260709 13:02:36.094460  9447 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:36.095172  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.095927  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:36.096352  9441 raft_consensus.cc:2410] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:36.096696  9447 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.102161 15614 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.080775  8898 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.105963  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:36.112541 15054 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.091885  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:36.120976  9446 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.090222 15645 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:36.100564  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.123545 15630 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.118984  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:36.136456 15428 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:36.137248 15428 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.136529  9440 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:36.167330  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61772.8us
I20260709 13:02:36.171391  8898 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:36.172241 15632 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.175148  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2463) took 1379 ms. Trace:
I20260709 13:02:36.178738 15559 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:36.081847 15593 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.085058 15617 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:36.187201  9447 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:36.191013  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 79536.2us
I20260709 13:02:36.193454 15419 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.194052  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.194363 15419 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.195091  9441 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:36.187435 15304 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.194378  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.198848 15506 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.204861 15630 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.205859  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.207275  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.205687 15630 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.209306 15617 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.208029  8745 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:36.212158 15617 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.213631 15419 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.179703 15559 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.220736  9446 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
W20260709 13:02:36.228638  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.080729 15639 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.232892  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.224053 15115 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:36.175959  8824 rpcz_store.cc:276] 0709 13:02:34.795471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.831638 (+ 36167us) service_pool.cc:224] Handling call
0709 13:02:34.832139 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.940813 (+108674us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:34.948001 (+  7188us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.446854 (+498853us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b180048cc40. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:02:35.447061 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.447107 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:35.447153 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.168713 (+721560us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003e2bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.172099 (+  3386us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.174595 (+  2496us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":132,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":3532,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"spinlock_wait_cycles":252471936,"thread_start_us":42386,"threads_started":1,"wal-append.queue_time_us":194156}
I20260709 13:02:36.239629 15115 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.240991 15630 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:36.266299  9414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.016s	sys 0.024s
I20260709 13:02:36.226454 15593 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.270603  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:36.247035 15671 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:36.272274  9414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.016s	sys 0.025s
I20260709 13:02:36.261184 15115 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.276609  8745 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.281633 15428 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.283857  9445 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:36.293238 15671 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.295611  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2462) took 1502 ms. Trace:
I20260709 13:02:36.296197  8823 rpcz_store.cc:276] 0709 13:02:34.793319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.793943 (+   624us) service_pool.cc:224] Handling call
0709 13:02:36.295128 (+1501185us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540028b754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:36.295353 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180612480}
I20260709 13:02:36.296646 15540 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.264079 15506 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:36.297335 15671 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.306704  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2699) took 2852 ms. Trace:
I20260709 13:02:36.307143  8893 rpcz_store.cc:276] 0709 13:02:33.451025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.458617 (+  7592us) service_pool.cc:224] Handling call
0709 13:02:34.227320 (+768703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.228177 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.292942 (+2064765us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b540045a6d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:36.303334 (+ 10392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1730618240}
I20260709 13:02:36.308302 15054 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:36.308914  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.311117  8898 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.316607  8713 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:36.317440 15537 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.345s	user 0.005s	sys 0.008s
I20260709 13:02:36.317751 15506 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:36.308640 15590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.029s	sys 0.035s
I20260709 13:02:36.319674  8893 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:36.321063  9440 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:36.323807  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.326354  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2443) took 3124 ms. Trace:
I20260709 13:02:36.326617  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.326856  8820 rpcz_store.cc:276] 0709 13:02:33.195894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.300693 (+104799us) service_pool.cc:224] Handling call
0709 13:02:33.300999 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.304891 (+  3892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.304935 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.305466 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.562357 (+256891us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:33.562399 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.562467 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.316389 (+2753922us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:36.319543 (+  3154us) raft_consensus.cc:1685] finished
0709 13:02:36.319567 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.320499 (+   932us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1564,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"spinlock_wait_cycles":3575680,"thread_start_us":256562,"threads_started":1,"wal-append.queue_time_us":631084}
I20260709 13:02:36.305819 15540 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.330236  8741 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:36.334266 15304 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:36.335430  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.336287 15639 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.332782 15428 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:36.337814 15590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.029s	sys 0.036s
W20260709 13:02:36.339485 15357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.013s	sys 0.038s
I20260709 13:02:36.346436 15414 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.354791  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:36.355046 15357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.014s	sys 0.039s
W20260709 13:02:36.363080  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.356750  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 729994us
I20260709 13:02:36.343479 15633 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:36.373714  9448 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:36.373868 15633 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:36.375317  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 58871.6us
I20260709 13:02:36.377027 15633 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:36.365126  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.370289  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.357511 15576 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.378624 15674 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:02:36.380843 15593 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.381670 15576 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.384004 15576 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.384402 15506 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:36.389577 15604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.011s	sys 0.014s
W20260709 13:02:36.390259 15604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.012s	sys 0.014s
I20260709 13:02:36.390583 15674 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:36.391278  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:36.391604 15559 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.392180 15559 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.392306 15593 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.392915 15559 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.393834 15593 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.393007 15676 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.395080 15676 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.399178 15593 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.399889 15676 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.399967 15280 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:36.400679 15630 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:36.391294 15604 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:36.401895  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.402213  8889 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:36.402173 15604 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.403195 15584 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.403563 15428 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.403770 15584 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.404277 15428 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:36.405149  8862 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:36.406351 15280 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:36.406216 15428 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.406910 15280 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:36.407827  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.408532 15604 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.408140 15280 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.412736 15357 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:36.413493 15639 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:36.399433 15228 consensus_meta.cc:287] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.801s	user 0.007s	sys 0.011s
I20260709 13:02:36.391168 15674 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:36.420898 15540 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.421725 15506 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:36.422410  8862 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:36.423281  8740 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:36.430899  8898 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:36.436707  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.437482  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.437652 15228 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:36.438651  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2732) took 1231 ms. Trace:
I20260709 13:02:36.438800  8747 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:36.437528  9445 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:36.449138 15506 raft_consensus.cc:2804] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:36.449262 15674 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.449805  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.439116  8895 rpcz_store.cc:276] 0709 13:02:35.207083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.207289 (+   206us) service_pool.cc:224] Handling call
0709 13:02:35.207640 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.585252 (+377612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.438176 (+852924us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400248e54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.438462 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861636352}
I20260709 13:02:36.456053  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.450070 15506 raft_consensus.cc:697] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:36.423241 15115 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.462029  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.464018  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.466713 15115 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.468508  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:36.469231  8895 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:36.460676 15559 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.483526  9446 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
W20260709 13:02:36.489449 15578 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:02:36.490620 15674 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.491745  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1186) took 2141 ms. Trace:
I20260709 13:02:36.492188  8891 rpcz_store.cc:276] 0709 13:02:34.349789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.349898 (+   109us) service_pool.cc:224] Handling call
0709 13:02:34.350186 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.350578 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:34.407755 (+ 57177us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.425946 (+ 18191us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.426384 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.426500 (+   116us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:34.426530 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.427068 (+   538us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.366651 (+1939583us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18000cc460. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:36.367797 (+  1146us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18000cc460. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:36.488611 (+120814us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:36.491230 (+  2619us) raft_consensus.cc:1685] finished
0709 13:02:36.491260 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.491542 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79476,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":3678,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7811,"spinlock_wait_cycles":42252672,"thread_start_us":56355,"threads_started":2,"wal-append.queue_time_us":22265}
I20260709 13:02:36.494707  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:36.461100 15228 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.499378  8745 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
W20260709 13:02:36.496567 15588 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.767s	user 0.016s	sys 0.007s
I20260709 13:02:36.501530  8891 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.501827  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.502571 15584 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.504918 15676 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.498916 15630 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.505671 15630 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.508741 15630 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.509312 15676 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.509766 15676 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.510145  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.511263  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51668 (request call id 2670) took 1834 ms. Trace:
I20260709 13:02:36.511633  8744 rpcz_store.cc:276] 0709 13:02:34.677067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.677228 (+   161us) service_pool.cc:224] Handling call
0709 13:02:36.510907 (+1833679us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001d9a54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:36.511093 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326212992}
I20260709 13:02:36.512887  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.501866 15559 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.514349 15559 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.515141  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2738) took 1267 ms. Trace:
I20260709 13:02:36.515493  8890 rpcz_store.cc:276] 0709 13:02:35.247724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.452802 (+205078us) service_pool.cc:224] Handling call
0709 13:02:35.453178 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.563798 (+110620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.514732 (+950934us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b54001806d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.514975 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99154176}
I20260709 13:02:36.516541 15559 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.516743  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.517341  8740 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:36.518654  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.519505  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.545773  8740 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:36.545974  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.546658  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.521065  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.548391  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.522284  8891 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:36.522511  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 3026) took 1713 ms. Trace:
I20260709 13:02:36.523407 15584 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.549654  8748 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:36.550046 15584 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.550748 15599 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:36.551291  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.551445 15599 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.552145  8748 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:36.552654 15584 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.553418  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.549661  8822 rpcz_store.cc:276] 0709 13:02:34.808868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.834505 (+ 25637us) service_pool.cc:224] Handling call
0709 13:02:34.834822 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.125973 (+291151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.126030 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.126081 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:35.126146 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.521997 (+1395851us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400074c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.522081 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.522306 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669154176}
I20260709 13:02:36.528070  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:36.529397 15681 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:36.530408 15506 consensus_queue.cc:237] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.532596 15676 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.556195  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.553442 15633 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.558018 15676 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.558766 15115 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.498924 15228 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.559422 15115 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.559713  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:36.560431 15428 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:36.560990 15428 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.536864  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.560487  9446 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.561878  8745 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.559662 15228 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.539031  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.563872  8742 raft_consensus.cc:2376] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21. Re-sending same reply.
I20260709 13:02:36.564904  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.561451 15428 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:36.567046  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.567476 15503 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:36.568060 15503 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:36.570091  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:36.572269 15496 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.574884 15600 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:36.539143  8641 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 286) took 42482 ms (client timeout 15000 ms). Trace:
I20260709 13:02:36.532457  8644 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 13, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 13 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:36.542265  8894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:02:36.554332  8748 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:36.558432 15633 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.534196  8890 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
W20260709 13:02:36.533955 15625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.003s	sys 0.035s
I20260709 13:02:36.535048  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.585575  8743 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:36.581009  8745 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:36.533382  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.587581  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:36.584797  8894 raft_consensus.cc:606] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:02:36.593040  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.592628  8644 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 11 to 14, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 14 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:36.594513  8644 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 11, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 11 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:36.595562  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.596840  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:36.597461  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.597756 15115 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.598124  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:36.598444 15541 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.630s	user 0.008s	sys 0.006s
I20260709 13:02:36.599388  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.599939  8748 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.553229 15584 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:36.601939 15584 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.604806 15584 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:36.587230 15625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.003s	sys 0.036s
I20260709 13:02:36.605944  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.575660 15600 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.613994  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:36.606519  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.615001  8740 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:36.617549  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.629716  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:36.630810  8743 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:36.634209  9446 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:36.634859  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:36.635671  8894 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:36.636185  8891 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:36.636368  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.638243  8742 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:36.640138 15600 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.645437 15676 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:36.646607  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:36.646615 15676 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:36.648396 15676 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:36.649677 15676 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.652945  8747 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:36.653263 15228 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.655293  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.655730 15228 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.653862 15676 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.658324  8644 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 10 to 11, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 11 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:36.660516  8895 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:36.663493 15584 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.657275  8744 raft_consensus.cc:2368] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19.
I20260709 13:02:36.664202 15584 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.568991 15503 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.666383 15584 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.668907  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:36.670596  8895 raft_consensus.cc:2368] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18.
I20260709 13:02:36.673405  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.674767  8740 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:36.669041  8748 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:36.673075  8895 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:36.687577  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.688652  8823 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:36.689000 15540 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.689680 15540 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:36.690310  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:36.690824 15540 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.691253  9446 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.693732 15633 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:36.694839  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.696429  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.677713  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.699790  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:36.656877  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:36.704516  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47722.1us
I20260709 13:02:36.708426  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2929) took 1414 ms. Trace:
I20260709 13:02:36.708958  8895 rpcz_store.cc:276] 0709 13:02:35.285335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.671613 (+1386278us) service_pool.cc:224] Handling call
0709 13:02:36.672540 (+   927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.698293 (+ 25753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.699435 (+  1142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.715364  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.702159  8822 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:36.719550  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2920) took 1438 ms. Trace:
I20260709 13:02:36.660454  8644 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 8 to 9, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 9 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:36.722838  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:36.721318  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.720173  8894 rpcz_store.cc:276] 0709 13:02:35.265586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.592622 (+1327036us) service_pool.cc:224] Handling call
0709 13:02:36.632393 (+ 39771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.703458 (+ 71065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.703660 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.733167  9446 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:36.657907 15115 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:36.735567 15115 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.736074 15115 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:36.741464  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.744458  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.745692  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.716485 15506 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:36.766268  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.771660  8824 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:36.766710  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:36.772740  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.771971  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.743485 15228 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:36.794191  8895 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 22: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:36.777611  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:36.797732  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 57164.4us
W20260709 13:02:36.798048  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 171656us
I20260709 13:02:36.798593  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.798959  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:36.798772  8894 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:02:36.799925  8894 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 9
I20260709 13:02:36.800293  8894 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:02:36.800606  8894 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.801410  8817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1165) took 3568 ms. Trace:
I20260709 13:02:36.801518  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2930) took 1514 ms. Trace:
I20260709 13:02:36.802474  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.801414 15611 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:36.805619  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:36.799018 15428 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.807797 15615 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:36.808022 15280 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:36.808951 15428 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:36.808837 15280 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:36.810549 15280 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:36.810931 15280 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:36.577277  8641 rpcz_store.cc:269] 0709 13:01:54.056514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.056863 (+   349us) service_pool.cc:224] Handling call
0709 13:02:36.538833 (+42481970us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:20.335996 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:20.336733 (+   737us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:20.336791 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:02:20.336812 (+    21us) tablet.cc:662] Decoding operations
0709 13:02:20.442385 (+105573us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:20.442512 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:20.442550 (+    38us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:02:20.624370 (+181820us) tablet.cc:701] Row locks acquired
0709 13:02:20.624424 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:02:20.624627 (+   203us) write_op.cc:270] Start()
0709 13:02:20.624784 (+   157us) write_op.cc:276] Timestamp: P: 1783602140624556 usec, L: 0
0709 13:02:20.624821 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:02:20.671429 (+ 46608us) log.cc:844] Serialized 10682 byte log entry
0709 13:02:21.269216 (+597787us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b
0709 13:02:23.087346 (+1818130us) op_driver.cc:464] REPLICATION: finished
0709 13:02:23.602849 (+515503us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f9f05ef0
0709 13:02:24.133543 (+530694us) write_op.cc:301] APPLY: starting
0709 13:02:24.206677 (+ 73134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:24.355495 (+148818us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cc32a 00007f90f6451f14 00007f90f6451ec3 00007f90f6451e38 00007f90f6451df3 00007f90f64510dc 00007f90f647ae16 00007f90f6462cbb 00007f90f644d39d 00007f90f6450d75
0709 13:02:25.276170 (+920675us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:25.276232 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:25.592339 (+316107us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:25.593143 (+   804us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:02:25.593241 (+    98us) write_op.cc:312] APPLY: finished
0709 13:02:25.651661 (+ 58420us) log.cc:844] Serialized 236 byte log entry
0709 13:02:25.862280 (+210619us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b
0709 13:02:26.157096 (+294816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:26.181930 (+ 24834us) write_op.cc:454] Released schema lock
0709 13:02:26.182027 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:26.182567 (+   540us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:26.182599 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":419787,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":912149,"apply.run_cpu_time_us":82293,"apply.run_wall_time_us":2107813,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116933,"num_ops":27,"prepare.queue_time_us":316211,"prepare.run_cpu_time_us":27504,"prepare.run_wall_time_us":1204720,"raft.queue_time_us":936022,"raft.run_cpu_time_us":55115,"raft.run_wall_time_us":1478807,"replication_time_us":2432399,"spinlock_wait_cycles":499490560,"thread_start_us":2890014,"threads_started":5,"wal-append.queue_time_us":1510704,"wal-append.run_cpu_time_us":102144,"wal-append.run_wall_time_us":3265972}]]}
I20260709 13:02:36.763625 15599 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:36.803121  8894 rpcz_store.cc:276] 0709 13:02:35.288019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.798026 (+1510007us) service_pool.cc:224] Handling call
0709 13:02:36.798359 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.801233 (+  2874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.801445 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.815995  8644 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 11, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 11 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:36.828559  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2723) took 1938 ms. Trace:
I20260709 13:02:36.829092  8892 rpcz_store.cc:276] 0709 13:02:34.889577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.891018 (+  1441us) service_pool.cc:224] Handling call
0709 13:02:34.891407 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.656919 (+765512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.827982 (+1171063us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b540046e954. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.828379 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1730738432}
I20260709 13:02:36.793510 15584 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:36.795007 15506 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:36.801890  8817 rpcz_store.cc:276] 0709 13:02:33.129838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.285544 (+155706us) service_pool.cc:224] Handling call
0709 13:02:33.286622 (+  1078us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:33.287616 (+   994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.287659 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:33.310095 (+ 22436us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.566467 (+256372us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.566510 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.566581 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.597490 (+3030909us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:36.606774 (+  9284us) raft_consensus.cc:1685] finished
0709 13:02:36.606796 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.698533 (+ 91737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":363,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":22272,"spinlock_wait_cycles":9395456,"thread_start_us":226515,"threads_started":1,"wal-append.queue_time_us":524010}
W20260709 13:02:36.842151  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 238476us
I20260709 13:02:36.847024 15601 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.847613  8895 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:36.848220  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2931) took 1558 ms. Trace:
I20260709 13:02:36.848531  8895 rpcz_store.cc:276] 0709 13:02:35.290491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.821598 (+1531107us) service_pool.cc:224] Handling call
0709 13:02:36.847379 (+ 25781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.848001 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.848172 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:36.850904 15115 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:36.852095  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2722) took 1962 ms. Trace:
I20260709 13:02:36.852557  8787 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:36.852648  8889 rpcz_store.cc:276] 0709 13:02:34.889248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.899357 (+ 10109us) service_pool.cc:224] Handling call
0709 13:02:34.927570 (+ 28213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.402764 (+1475194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.851861 (+449097us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400047ed4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.852040 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3390571392}
I20260709 13:02:36.853837 15599 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.854120  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.854408 15599 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:36.855405 15538 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.894s	user 0.004s	sys 0.000s
I20260709 13:02:36.855870 15599 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.858210  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.854163  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1182) took 1644 ms. Trace:
I20260709 13:02:36.857386  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2439) took 3870 ms. Trace:
I20260709 13:02:36.855882  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.866065  8889 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:36.860606  8819 rpcz_store.cc:276] 0709 13:02:35.209425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.329454 (+120029us) service_pool.cc:224] Handling call
0709 13:02:35.329765 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.431244 (+101479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.431277 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.431330 (+    53us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:35.431364 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.853788 (+1422424us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003e30d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:36.853853 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.854016 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232927616}
I20260709 13:02:36.875739 15115 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:36.877745  8895 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 14. Current term is 16. Ops: []
I20260709 13:02:36.883642  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:36.887894  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.885677  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:36.898631  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.861680  8816 rpcz_store.cc:276] 0709 13:02:32.986525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.109780 (+123255us) service_pool.cc:224] Handling call
0709 13:02:33.110055 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.199869 (+ 89814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.199915 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.200478 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.268161 (+ 67683us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b48002826d8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fb3f8aa1 00007f90fb3efb37 00007f90fb3ef9ca 00007f90fb4e726f 00007f90fb4e6f37 00007f90fb4fb55c 00007f90fb4fb508 00007f90fb4fb488 00007f90fb4fb443 00007f90fb4fa6ec 00007f90fa424a96
0709 13:02:33.532432 (+264271us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:33.532479 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.532548 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.854498 (+3321950us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:36.856915 (+  2417us) raft_consensus.cc:1685] finished
0709 13:02:36.856948 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.857172 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57412,"raft.queue_time_us":66644,"raft.run_cpu_time_us":2460,"raft.run_wall_time_us":59122,"spinlock_wait_cycles":107995776,"thread_start_us":213425,"threads_started":1,"wal-append.queue_time_us":472807}
I20260709 13:02:36.902107  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.905488 15595 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:36.906177 15595 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:36.910229  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.908322  9441 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:02:36.898940  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.930331  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.931629  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:36.937186  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:36.951119  8817 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:02:36.951982 15626 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:36.953948  8816 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:36.957373 15687 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:02:36.960536 15613 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:36.961086  8749 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:36.961311 15613 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:36.962482  8749 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:36.963878 15613 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:36.953035 15675 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.589s	user 0.000s	sys 0.003s
I20260709 13:02:36.965696  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.978463  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:36.954792 15506 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:36.982280  8821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 1157) took 4541 ms (client timeout 1788 ms). Trace:
W20260709 13:02:36.982807  8821 rpcz_store.cc:269] 0709 13:02:32.440727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.483946 (+ 43219us) service_pool.cc:224] Handling call
0709 13:02:36.981792 (+4497846us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001f04d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:36.982061 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4356146560}
W20260709 13:02:36.983976  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 124757us
I20260709 13:02:36.989830  8891 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:36.991792  8894 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:36.993423  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:36.990290  8644 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 13 to 15, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 15 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:37.002935  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.006588  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:37.007390  9447 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.976773 15280 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.024672  8819 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:02:37.026897  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:37.035964  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.010792 15613 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:02:37.050226  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.052086  8819 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: [19.17-19.17]
W20260709 13:02:37.060005  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61902.7us
W20260709 13:02:37.062182  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 189292us
I20260709 13:02:37.039512 15492 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:37.061631  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 174352us
I20260709 13:02:37.039455 15546 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.066470  8824 raft_consensus.cc:2393] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:37.068364  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2739) took 1458 ms. Trace:
I20260709 13:02:37.069473  8889 rpcz_store.cc:276] 0709 13:02:35.603760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.015854 (+1412094us) service_pool.cc:224] Handling call
0709 13:02:37.036082 (+ 20228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.045127 (+  9045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.045169 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.061643 (+ 16474us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:37.061685 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.061739 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.061839 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:37.072136  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.086455  9441 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:37.086805  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.085582  8787 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:37.033079 15595 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.098893  9447 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:37.092434 15492 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.102036  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2741) took 1497 ms. Trace:
I20260709 13:02:37.101914 15492 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.102453  8893 rpcz_store.cc:276] 0709 13:02:35.604336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.308439 (+704103us) service_pool.cc:224] Handling call
0709 13:02:37.101830 (+793391us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b540015de54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:37.101951 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16875264}
I20260709 13:02:37.104099  8893 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.104467 15492 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.104750  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2937) took 1499 ms. Trace:
I20260709 13:02:37.094450 15633 consensus_queue.cc:1059] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
W20260709 13:02:37.106463  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 332682us
I20260709 13:02:37.107019 15266 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:37.107460 15266 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.108464 15266 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:37.109504  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.110034  8740 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:37.112165 15601 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:37.112790  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 260189us
I20260709 13:02:37.113736 15266 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.114311 15280 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:37.114552  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.084326 15626 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:37.110536 15674 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:37.115752  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52899.5us
I20260709 13:02:37.116113 15674 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.094718 15604 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.116549  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.110146 15595 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.105118  8893 rpcz_store.cc:276] 0709 13:02:35.605192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.103518 (+1498326us) service_pool.cc:224] Handling call
0709 13:02:37.103911 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.104497 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.104674 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.095542  8644 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 8 to 12, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:37.146435  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66227.9us
W20260709 13:02:37.148660  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 99786.6us
I20260709 13:02:37.158222  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:37.116642 15674 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.175779  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:37.177256  8747 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:02:37.175907 15628 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:37.181146  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66713.1us
I20260709 13:02:37.181488  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:37.183192  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.184594  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.185462 15664 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.186378 15620 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:37.113181 15601 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.192128 15620 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:37.193400  8745 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:37.193429  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.195006  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2936) took 1590 ms. Trace:
I20260709 13:02:37.195314  8898 rpcz_store.cc:276] 0709 13:02:35.604919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.156168 (+1551249us) service_pool.cc:224] Handling call
0709 13:02:37.194948 (+ 38780us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.196533  8898 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:37.197240  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80810.2us
I20260709 13:02:37.198129  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.198612  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1221) took 2021 ms. Trace:
I20260709 13:02:37.198987  9443 rpcz_store.cc:276] 0709 13:02:35.177435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.177554 (+   119us) service_pool.cc:224] Handling call
0709 13:02:35.177848 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.458483 (+280635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.157359 (+1698876us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54006801d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:37.198448 (+ 41089us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644182400}
I20260709 13:02:37.185566  9449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1300) took 1872 ms. Trace:
W20260709 13:02:37.201335  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 49323.5us
I20260709 13:02:37.200249  9449 rpcz_store.cc:276] 0709 13:02:35.313167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.313733 (+   566us) service_pool.cc:224] Handling call
0709 13:02:35.314007 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.314227 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:02:35.403351 (+ 89124us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.848361 (+445010us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b18002fde20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:36.991596 (+1143235us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b18002fde20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf
0709 13:02:37.011570 (+ 19974us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.146425 (+134855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.154691 (+  8266us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:02:37.159042 (+  4351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.185201 (+ 26159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.185391 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44564,"prepare.queue_time_us":267773,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":1349,"raft.queue_time_us":29504,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":4955,"spinlock_wait_cycles":14252544,"thread_start_us":409955,"threads_started":4,"wal-append.queue_time_us":244337,"wal-append.run_cpu_time_us":1171,"wal-append.run_wall_time_us":1009664}
I20260709 13:02:37.204829  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.205122  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.186239 15664 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.188473  8889 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:37.205803  8747 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:37.204838  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.206341  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2740) took 1602 ms. Trace:
I20260709 13:02:37.207059  8747 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:37.206734  8889 rpcz_store.cc:276] 0709 13:02:35.604044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.084590 (+1480546us) service_pool.cc:224] Handling call
0709 13:02:37.188044 (+103454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.206063 (+ 18019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.206238 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:37.207536  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.156823 15604 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.203773 15228 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:02:37.198052  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2731) took 2181 ms. Trace:
W20260709 13:02:37.226112 15694 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:02:37.209267  8747 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:37.232921 15698 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:37.233831  8890 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:37.235173  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2927) took 1953 ms. Trace:
I20260709 13:02:37.235520  8890 rpcz_store.cc:276] 0709 13:02:35.282000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.594221 (+1312221us) service_pool.cc:224] Handling call
0709 13:02:36.594599 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.234334 (+639735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.234876 (+   542us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400249354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:37.235071 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469569792}
I20260709 13:02:37.236961  8890 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.238279  8893 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:02:37.202167  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.248499  8745 raft_consensus.cc:1240] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:37.252514  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.253163  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.243513  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2932) took 1949 ms. Trace:
I20260709 13:02:37.253856  8894 rpcz_store.cc:276] 0709 13:02:35.294820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.826341 (+1531521us) service_pool.cc:224] Handling call
0709 13:02:36.990624 (+164283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.998274 (+  7650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.243161 (+244887us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400048dd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:37.243437 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303278464}
W20260709 13:02:37.237910  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44808us
I20260709 13:02:37.237582  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.214182  8818 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:37.214742  8824 raft_consensus.cc:1240] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.216849  9449 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.219241  8740 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:37.219499  9444 raft_consensus.cc:1240] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:37.221791  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.213011  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.224310  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2935) took 1912 ms. Trace:
W20260709 13:02:37.220819  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 381972us
I20260709 13:02:37.220019  9448 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.213797  8889 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:37.264843  8895 rpcz_store.cc:276] 0709 13:02:35.311983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.849267 (+1537284us) service_pool.cc:224] Handling call
0709 13:02:36.849650 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.878276 (+ 28626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.223984 (+345708us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5400030cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:37.224204 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64148224}
I20260709 13:02:37.213884  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:37.260468  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61684.5us
W20260709 13:02:37.204718 15688 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:02:37.190116 15611 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:37.228264 15696 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:02:37.215596  8897 rpcz_store.cc:276] 0709 13:02:35.016031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.016168 (+   137us) service_pool.cc:224] Handling call
0709 13:02:35.016441 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.473924 (+457483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.191018 (+1717094us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54000843d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:37.197887 (+  6869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050996992}
I20260709 13:02:37.293174  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.269625  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.165175 15626 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.159253 15492 raft_consensus.cc:3055] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:37.215996 15689 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.287s	user 0.001s	sys 0.001s
I20260709 13:02:37.295802  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.296214  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:37.290639  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.285562  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2743) took 1638 ms. Trace:
I20260709 13:02:37.297928  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.188594 15676 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:37.297766  8893 rpcz_store.cc:276] 0709 13:02:35.646863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.128559 (+1481696us) service_pool.cc:224] Handling call
0709 13:02:37.237956 (+109397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.250703 (+ 12747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.274449 (+ 23746us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b5400249350. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:37.285463 (+ 11014us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112199552}
I20260709 13:02:37.300163 14821 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:37.221073 15584 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:37.301014 15332 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 13:02:37.301756 15584 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.303299 15620 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.303869 15650 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.304453 15620 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.213480 15599 consensus_queue.cc:1059] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
W20260709 13:02:37.305949 15700 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:37.213496 15633 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:37.220736 15428 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.306602 15633 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.215507 15546 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:37.307224 15428 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.209658 15563 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 13:02:37.307801 15546 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:37.308486 15563 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:37.308848 15546 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.295305 15492 raft_consensus.cc:740] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.310333 15546 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.310506 15492 consensus_queue.cc:260] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.311970 15492 raft_consensus.cc:3060] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:37.312978 15611 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.209672 15496 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:37.297173  9441 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:37.211602 15604 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:37.317664 15428 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:37.318675 15599 raft_consensus.cc:3055] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:37.319085 15599 raft_consensus.cc:740] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.319396  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2417) took 1115 ms. Trace:
I20260709 13:02:37.320093 15599 consensus_queue.cc:260] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.319779  8743 rpcz_store.cc:276] 0709 13:02:36.203754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.637686 (+433932us) service_pool.cc:224] Handling call
0709 13:02:36.638632 (+   946us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:36.639143 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.639172 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:36.639817 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 13:02:36.761466 (+121649us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:36.766759 (+  5293us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.768870 (+  2111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.768948 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.100426 (+331478us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b180048d240. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90fa3a358f 00007f90f33e2571 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd
0709 13:02:37.215177 (+114751us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:37.319152 (+103975us) raft_consensus.cc:1685] finished
0709 13:02:37.319203 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.319340 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11587,"prepare.queue_time_us":1005,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":6607,"raft.queue_time_us":5101,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":5631,"spinlock_wait_cycles":22274304,"thread_start_us":120494,"threads_started":1,"wal-append.queue_time_us":203039}
W20260709 13:02:37.321621  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 123609us
I20260709 13:02:37.322454  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.287433 15228 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.267206 15628 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:37.326166 15459 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.325861 15628 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.326583 15683 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:02:37.327346  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.308854 15563 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:37.329517 15563 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.330603 15563 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.220764 15613 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:37.336148  8898 raft_consensus.cc:1240] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:37.337879  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:37.338379  9414 leader_election.cc:400] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19.
I20260709 13:02:37.338735  8895 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.338750  9414 leader_election.cc:403] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.295662 15596 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.351820  9448 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:37.315655 15633 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:37.336342 15656 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.269958 15540 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:02:37.374930  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.359009 15676 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:37.375810  8817 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:37.361162 15599 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.351233 15496 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:37.388640  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.391104  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48569.7us
W20260709 13:02:37.397609  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.373612 15664 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.367481 15613 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.406877  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.407348  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:37.412467  8748 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.408758  8747 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:37.368041 15596 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.413697  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.414297  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:37.422119  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 97008.6us
I20260709 13:02:37.426600  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.425192  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:37.416910 15618 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.431685 15642 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19.
I20260709 13:02:37.432754  9412 leader_election.cc:400] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18.
I20260709 13:02:37.433183  9412 leader_election.cc:403] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.441226 15613 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:37.442608  8818 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:37.443104  8818 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.444020 15506 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:37.444221  8818 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.430346  8743 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:02:37.429844  8819 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:37.446269  8743 pending_rounds.cc:85] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 12
I20260709 13:02:37.433779  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:37.447392  8746 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.383713 15656 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.450332  8743 pending_rounds.cc:107] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:02:37.450734  8743 raft_consensus.cc:2889] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.452059  8744 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:37.453824  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2933) took 2145 ms. Trace:
W20260709 13:02:37.456368  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 75734.7us
I20260709 13:02:37.457924  8818 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.448246 15664 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.497051  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:37.499089 15492 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:37.499531  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:37.499866  8818 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:37.500875 15604 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:37.459997 15653 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18.
I20260709 13:02:37.499581 15492 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.502714 15492 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.503657  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1195) took 2163 ms. Trace:
I20260709 13:02:37.504067  8891 rpcz_store.cc:276] 0709 13:02:35.340115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.030685 (+1690570us) service_pool.cc:224] Handling call
0709 13:02:37.057850 (+ 27165us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.250283 (+192433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.250424 (+   141us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.314221 (+ 63797us) log.cc:844] Serialized 52 byte log entry
0709 13:02:37.418239 (+104018us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:37.418284 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.418355 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.503428 (+ 85073us) raft_consensus.cc:1685] finished
0709 13:02:37.503451 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.503599 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189195,"raft.queue_time_us":179954,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":22361,"thread_start_us":91087,"threads_started":1,"wal-append.queue_time_us":174941}
I20260709 13:02:37.505833 15656 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:37.507493  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:37.512300  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 79768us
I20260709 13:02:37.513803 15601 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:37.515178 15601 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.515715 15601 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.516049 15615 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:37.516386 15683 consensus_queue.cc:1059] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:37.428344 15626 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.517302 15615 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:37.517508 15683 raft_consensus.cc:3055] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:37.517820 15615 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:37.517990 15683 raft_consensus.cc:740] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:37.417723 15611 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.519071 15683 consensus_queue.cc:260] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.519223 15615 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.520685 15615 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.522403 15683 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.528425 15559 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:02:37.532405 15698 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:02:37.532575 15709 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:37.534930  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.535821 15656 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:37.540035 15615 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.501343 15604 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:37.445979 15496 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.555065 15604 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.547849 15653 consensus_queue.cc:1059] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:37.566874  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.567750  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.568727  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.546877  8748 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:02:37.563541  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.559818  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58858.7us
I20260709 13:02:37.564591  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:37.574769  8891 raft_consensus.cc:2376] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17. Re-sending same reply.
I20260709 13:02:37.581991 15332 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:37.582430 15332 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:37.510368 15492 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.600530  9412 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.601963  8746 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:37.579942 15674 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.606362  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1211) took 1002 ms. Trace:
I20260709 13:02:37.584470 15601 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:37.609994  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.629014  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.606770  8749 rpcz_store.cc:276] 0709 13:02:36.604163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.964260 (+360097us) service_pool.cc:224] Handling call
0709 13:02:36.964586 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.115678 (+151092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.115710 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.116301 (+   591us) log.cc:844] Serialized 52 byte log entry
0709 13:02:37.334635 (+218334us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:37.334671 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.334735 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.606145 (+271410us) raft_consensus.cc:1685] finished
0709 13:02:37.606166 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.606306 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147393,"raft.queue_time_us":147535,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":217982,"threads_started":1,"wal-append.queue_time_us":297136}
I20260709 13:02:37.628619  8788 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.638782  8816 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:37.640838  8745 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:02:37.641417  8745 pending_rounds.cc:85] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 14
I20260709 13:02:37.641700  8745 pending_rounds.cc:107] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:02:37.641999  8745 raft_consensus.cc:2889] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.643899  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:37.646222  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.583285 15332 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.648614  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.645184  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:37.652842  8893 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:37.659746 15459 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:37.631490 15711 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:02:37.663663  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.664288  8895 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:37.665220  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.666508  8748 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:37.666656 15332 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.665786  8895 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:37.669090  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.669610  8895 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:37.661409 15653 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.678733  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.684104  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.684463 15653 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.685388 15546 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:37.685532  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.685925 15546 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.686648  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.687228  8898 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:37.687906 15546 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.688019  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.689934 15601 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.691054 15601 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:37.691383 15601 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:37.691896 15653 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.692252 15601 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.694811  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:37.695362 15716 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.265s	user 0.004s	sys 0.000s
W20260709 13:02:37.696233  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 67730.4us
I20260709 13:02:37.699945  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.682240  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:37.703186  9441 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:37.697028 15601 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.695804  8895 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:37.705163  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:37.704082  8863 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:37.706621  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.709141  8740 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:37.709852  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:37.710593  8893 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.706517 15280 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:37.706610 15419 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:02:37.713330 15427 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.713461 15613 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:37.713801 15642 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:37.713910 15427 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.714443 15419 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.715545 15268 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:37.716084 15496 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:37.716440 15496 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:37.717046 15559 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:02:37.712779  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.714349 15642 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:37.713241 15280 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.719352 15642 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.719687 15280 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:37.717296 15496 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:37.727403  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 51072.8us
I20260709 13:02:37.729671  8898 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.675124 15459 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:37.727089  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:37.741869  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.746762  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:37.747767  9446 raft_consensus.cc:2393] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:37.743319  8787 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:37.754962  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:37.755187  9445 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:37.680164 15596 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.767855  9444 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:37.769110  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.776525  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.777524  8898 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 14: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:37.755622 15584 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:37.756134 15611 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:02:37.788084  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 86133.9us
I20260709 13:02:37.786859  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:37.766479 15595 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.793395  8889 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.797762  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:37.803604  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.774037 15596 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:37.760847 15633 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:02:37.813423  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:37.808977 15266 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:37.788213 15611 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.767532 15588 consensus_queue.cc:1059] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:37.811751  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:37.834321  8897 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:37.829437  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:37.824388  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:37.839206  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.846657 15646 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:37.795310 15584 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:37.871368  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:37.872432  8818 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:37.873808  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:37.874686  8897 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:37.823967 15611 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:37.879832  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 202818us
I20260709 13:02:37.814666 15633 consensus_queue.cc:1243] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:37.805866 15595 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.893018  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:37.882592  8817 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:37.890221  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.882452  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:37.894593  8823 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:37.875238 15725 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:37.858548 15546 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.905117  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.874661 15674 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:37.896409  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 84751.5us
I20260709 13:02:37.874619 15492 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:37.895603  8824 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:37.896237  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:37.870662 15266 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:37.455224  8892 rpcz_store.cc:276] 0709 13:02:35.308770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.830151 (+1521381us) service_pool.cc:224] Handling call
0709 13:02:36.830651 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:36.831186 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:36.837589 (+  6403us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.220543 (+382954us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b1800412120. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90fa3a358f 00007f90f33e2571 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd
0709 13:02:37.232083 (+ 11540us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:37.282214 (+ 50131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.290259 (+  8045us) log.cc:844] Serialized 26 byte log entry
0709 13:02:37.290454 (+   195us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:37.452878 (+162424us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.453416 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.453515 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.453583 (+    68us) raft_consensus.cc:1685] finished
0709 13:02:37.453600 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.453755 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255666,"prepare.queue_time_us":16725,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":194382,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":24286,"spinlock_wait_cycles":61511296,"thread_start_us":323364,"threads_started":2,"wal-append.queue_time_us":210592}
W20260709 13:02:37.896379  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 130688us
I20260709 13:02:37.900256 15718 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:37.949896  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:37.942883  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46809.5us
I20260709 13:02:37.822808 15459 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.927135 15266 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:37.968843  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.971056  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:37.970276  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:37.974090  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.974253  8820 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
W20260709 13:02:37.975023 15720 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:02:37.975423 15724 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:02:37.970727 15266 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.977561 15734 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:37.980019 15719 log.cc:927] Time spent T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:02:37.981487  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:37.981689 15723 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 13:02:37.981524  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.916267 15546 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.902518 15642 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.951103 15492 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.963393 15718 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:37.927693 15604 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:37.994392 15584 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:37.943823 15503 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:02:38.010452  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:37.956362 15280 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.021227  8787 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.021647  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:37.967609 15711 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:38.024957  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.016256 14821 raft_consensus.cc:3055] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:38.030774 15726 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:38.030855  9444 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:02:38.031201 15722 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:02:38.038108 15710 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.574s	user 0.006s	sys 0.000s
I20260709 13:02:38.079547  8897 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:38.040635 15633 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:38.023449 15740 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:02:38.041509 15427 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:38.042343 15736 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:02:38.042963  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.088960  8820 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:38.045565 15599 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:38.046190 15642 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:38.047343 15620 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.742s	user 0.005s	sys 0.005s
I20260709 13:02:38.049886  8823 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:38.056380 15595 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.106853  9449 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
W20260709 13:02:38.057217 15735 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:38.035422 15492 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:38.028633 15738 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:02:38.082964 15633 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.035952 15626 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:38.061148  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.078761  8819 raft_consensus.cc:2468] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:38.065613  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:38.081063  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.085146  8893 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:38.095872  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:38.100682  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.102114  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:38.102886  9448 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:38.033164 14821 raft_consensus.cc:740] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:38.110211  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:38.083140 15711 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.116178  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:38.117930 15427 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:38.118331 15427 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.118793 15427 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.121217  8896 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:38.087100 15419 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:02:38.086692 15613 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:38.126376  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.127884  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.113374  8818 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:38.130846  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.113435  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.132298 15266 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:38.115170  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49702.3us
I20260709 13:02:38.134442  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.114801 15506 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.114778 15411 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:02:38.116737 15503 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:38.135107  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.117476  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.129088  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.124958 15613 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.142971  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.145711 15546 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.145488 15613 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.145948 14821 consensus_queue.cc:260] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.147197  8821 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.142019  8891 pending_rounds.cc:85] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 8
I20260709 13:02:38.156790  8891 pending_rounds.cc:107] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:38.157070  8891 raft_consensus.cc:2889] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.114084 15633 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.111879 15492 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:38.134159 15604 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.163856  8890 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:38.171766  8817 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:02:38.172225  8817 pending_rounds.cc:85] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 14
I20260709 13:02:38.172696  8817 pending_rounds.cc:107] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:02:38.172991  8817 raft_consensus.cc:2889] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.181035  8895 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:38.167923  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:38.166198 14821 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:38.191869  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.166523 15492 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:38.195029 15747 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:38.195556 15266 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:38.197525  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54669.8us
W20260709 13:02:38.198686 15729 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:02:38.199498  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.204917 15622 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:38.206254 15738 raft_consensus.cc:993] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:02:38.201174  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:38.117450 15711 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:38.220225  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 120048us
I20260709 13:02:38.221375  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:38.226243  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.111681 15718 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.189646 15599 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:38.233510  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:38.241509  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:38.243840  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:38.218313 15595 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.255199  9446 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:38.228223 15115 raft_consensus.cc:1081] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:02:38.266083  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:38.232789 15748 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:38.197557 15740 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:02:38.240813 15656 raft_consensus.cc:3055] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:38.251838 15711 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.254144 15599 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:38.298447  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:38.241926 15427 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.309114  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:38.310480  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:38.280568 15656 raft_consensus.cc:740] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:38.280283 15280 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:38.276479 15563 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:38.309958  8740 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:38.319595  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:38.321590  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 103817us
I20260709 13:02:38.282608 15613 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.271708 15595 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.356655  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:38.330724 15563 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.364135 15563 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:38.364722 15268 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.364598 15633 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.366259  8818 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:38.371284  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.373700  8742 raft_consensus.cc:2393] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:38.387334  8743 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:38.394440  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:38.417793 15752 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:02:38.385782 15613 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:02:38.375509 15626 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:38.385934 15650 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:38.415948 15664 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.421073 15601 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.435500 15115 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:02:38.435745 15755 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:02:38.464322 15492 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:38.448182  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53748.5us
I20260709 13:02:38.386385 15611 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.421242 15546 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:38.454353 15601 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.428066 15332 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:38.387769 15599 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.516211 15599 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:38.436357 15633 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.470079  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:38.518240  8891 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:38.408002 15756 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:02:38.487722  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:38.446228  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.451318 15664 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.422423 15656 consensus_queue.cc:260] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.504663  8821 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:38.509336  8896 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:38.544185  8824 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:38.563920  8817 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:38.545733  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.565085  8824 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:38.567292  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.516567 15601 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.571019  8822 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:38.573460  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.485694 15595 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:38.569931  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 69721.8us
W20260709 13:02:38.584599  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 97030.3us
I20260709 13:02:38.593611  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:38.570652 15760 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.602803 15400 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.603384 15400 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.605885  8824 raft_consensus.cc:1275] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:38.606887  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:38.608297  8824 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:02:38.609524  8817 raft_consensus.cc:1240] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:38.608803 15633 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:38.610471  8817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 13:02:38.611078  8817 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:38.611469  8817 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:38.615566 15582 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:38.616128 15582 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.618073 15601 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.622097 15601 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.624548 15601 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:38.626081  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.628671 15650 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:38.629240 15650 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.629711 15650 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:38.620649  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:38.635614  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 199712us
W20260709 13:02:38.636909  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.639019  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.585216 15546 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.593251 15332 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:38.645123  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:38.646549  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 59228.3us
I20260709 13:02:38.650197 15492 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:38.651321 15742 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.671s	user 0.003s	sys 0.015s
I20260709 13:02:38.651822 15664 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.658691 15559 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:38.642179 15595 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:38.677906  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.679008  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.680476  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.648463 15601 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:38.650197 15611 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.689846  8864 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:38.646127 15546 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.697162  8864 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:38.683342 15332 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:38.684408 15588 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.691531 15739 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.693572 15747 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:38.707247 15770 log.cc:927] Time spent T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:02:38.704900 15400 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:38.689952 15559 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:38.695753 14821 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:02:38.715807 15611 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:38.728116 15611 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.717720 15664 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:02:38.729856 15664 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:38.730281 15664 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:38.730384 15611 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.717525 15628 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:02:38.731144 15664 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.732293 15664 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:38.721617 15756 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:38.733682 15756 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.735836 15756 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.737090 15604 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:38.706638 15563 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.737531 15604 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:38.738430 15604 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.739308 15563 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.739828 15604 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.743006  8817 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.721452 15582 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.752264  8820 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:38.757454  8817 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.763329 15650 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.711557 15718 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:38.773659  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 83267.3us
W20260709 13:02:38.773741  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 82142.9us
I20260709 13:02:38.780690  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.781159  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.783118 15563 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:38.782925 15738 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:38.783685  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:38.784546 15563 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.787217 15563 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.787585 15588 consensus_queue.cc:1048] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:38.787719 15563 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.788123 15738 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:38.805366  8745 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:02:38.805572 15728 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.698s	user 0.002s	sys 0.022s
I20260709 13:02:38.805987 15711 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.806627 15711 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.807751  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1214) took 1225 ms. Trace:
I20260709 13:02:38.808784 15711 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.808225  8747 rpcz_store.cc:276] 0709 13:02:37.581895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.583558 (+  1663us) service_pool.cc:224] Handling call
0709 13:02:37.589612 (+  6054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.604421 (+ 14809us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:37.604456 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.614516 (+ 10060us) log.cc:844] Serialized 26 byte log entry
0709 13:02:37.874718 (+260202us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:37.874767 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.874831 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.805146 (+930315us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b240011eaa8. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f342109d 00007f90f605310e 00007f90f606a88c 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221
0709 13:02:38.807381 (+  2235us) raft_consensus.cc:1685] finished
0709 13:02:38.807410 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.807659 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1023,"raft.queue_time_us":1536,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2208,"spinlock_wait_cycles":30735616,"thread_start_us":258507,"threads_started":1,"wal-append.queue_time_us":401970}
I20260709 13:02:38.811853 15582 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.824337  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:38.826709 15628 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:38.821525 15646 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.834378  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:38.879894 15747 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:38.880661 15747 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.883803 15747 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:38.875124 15563 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.877192 15780 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.897647 15780 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:38.899989 15780 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:38.887498 15779 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:38.889751 15546 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.886735 15626 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:38.902391 15626 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.905892 15626 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:38.876219 15646 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:38.913271 15646 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.918754 15595 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:38.919406 15595 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:38.927816  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 103612us
W20260709 13:02:38.953200 15774 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.357s	user 0.000s	sys 0.008s
I20260709 13:02:38.947932 15595 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:38.946048 15628 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:02:38.996034 15496 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:38.998279 15656 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.009214  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2481) took 1127 ms. Trace:
I20260709 13:02:39.009742  8822 rpcz_store.cc:276] 0709 13:02:37.882272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.883089 (+   817us) service_pool.cc:224] Handling call
0709 13:02:39.009022 (+1125933us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54003e3ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:39.009154 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529496832}
I20260709 13:02:39.014357 15595 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:39.014963 15595 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.017241 15595 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.024830  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.025720  8822 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:39.030603 15546 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:39.031513 15546 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.033569 15546 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.034008 15546 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:39.034487 15546 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.036322  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.036955  8745 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.037043  8741 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:02:39.040179  8815 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:39.043327  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.056672  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.059862  8743 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:02:39.064982  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43510.8us
I20260709 13:02:39.067037  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:39.067858  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.070012  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.071058  8816 raft_consensus.cc:2368] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 22. Current term is 23.
I20260709 13:02:39.071725  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.073632  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.074458  8749 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:39.068485  8741 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:39.078361  8746 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:39.036366 15546 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.092665  9415 leader_election.cc:400] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 22. Current term is 23.
I20260709 13:02:39.093243  9415 leader_election.cc:403] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:39.079924  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.074846 15622 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:39.107489  8745 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:39.108884  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.110090  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.110997  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.114456  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.115027  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.115260  8741 raft_consensus.cc:2393] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:39.116074  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.116649  8741 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:39.122279  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.123659  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.132045 15740 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:39.132861 15740 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.135694 15740 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.144503  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.078972 15628 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:39.147343  8749 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:39.147343  8818 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:39.082039 15563 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.148113  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:39.084796 15664 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:39.150888  8744 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:02:39.154242  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:39.160575 15650 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.095080 15601 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:39.108295 15622 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:39.156560  8818 raft_consensus.cc:2955] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 2 to 10, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:39.149464 15615 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:02:39.155296 15506 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:39.170658  8745 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:02:39.170022  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:02:39.187880  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.184784  8745 pending_rounds.cc:85] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 8
I20260709 13:02:39.188799  8741 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:39.189838  8889 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:39.189157  8745 pending_rounds.cc:107] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:02:39.190689  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.191052  8745 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.169539 15622 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:39.198388  8746 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:39.201401  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.201639  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.202381  8741 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:39.204169  8745 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:39.202109  8898 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:39.166008 15796 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:39.210948  8746 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:39.214450 15788 log.cc:927] Time spent T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.208s	user 0.000s	sys 0.002s
I20260709 13:02:39.163964 15563 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.216709 15563 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.220395  8817 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:39.195264 15626 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:39.221524 15626 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:39.221946 15626 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.169322 15601 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.153749 15628 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.228681  8743 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:39.236163  8743 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:39.230469  8817 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:39.164458 15664 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.232805  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.240326  8745 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:39.251238 15332 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:39.251742 15332 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:39.260253 15748 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.272780 15615 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 22. Current term is 23.
I20260709 13:02:39.305989  8741 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:02:39.308629  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 145661us
I20260709 13:02:39.260874 15748 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.321973  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2491) took 1260 ms. Trace:
I20260709 13:02:39.322619  8821 rpcz_store.cc:276] 0709 13:02:38.061594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.128815 (+ 67221us) service_pool.cc:224] Handling call
0709 13:02:39.321707 (+1192892us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b540028b754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:39.321894 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10193792}
I20260709 13:02:39.310599 15268 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.346856 15268 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.358377 15628 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:39.359834 15268 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:39.349419 15546 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.361598 15546 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.362564  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.363058 15506 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:39.363992 15546 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:39.364557 15793 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:02:39.371289 15622 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.351029 15656 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:39.374327 15656 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.376371  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.379096  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.380411  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:39.381254  8747 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:39.350643 15804 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.382432 15804 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.383451 15601 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.384784 15664 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:39.347527 15789 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.347306 15796 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.385538 15804 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:39.387372 15496 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.382438 15622 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:39.388099 15496 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.388484  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:39.413213  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:39.413942  8747 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:39.417996  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:39.421396  8821 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:39.410377 15496 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.411062 15789 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.431094 15626 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.432379 15633 consensus_queue.cc:1059] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:39.434096 15427 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.386267 15796 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:39.447418  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 218920us
I20260709 13:02:39.449260  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:39.439308 15656 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:39.511234 15772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.022s	sys 0.011s
I20260709 13:02:39.501050 15563 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.503971 15800 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:39.530853 15772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.022s	sys 0.011s
I20260709 13:02:39.506870 15506 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:39.507063 15115 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:39.534469  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 183475us
I20260709 13:02:39.538688  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:39.506629 15646 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:02:39.506958 15802 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:39.543949  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:39.546437 15808 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.536724 15563 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.571568 15818 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:39.575228 15818 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:39.576691 15764 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.963s	user 0.009s	sys 0.000s
I20260709 13:02:39.576182 15818 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.541671 15800 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.552088 15268 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:39.547950 15506 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.557370 15115 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.592147 15800 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.569876 15808 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.595342 15626 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.616912 15626 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.617609 15626 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.618511 15646 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:39.665349  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:39.665548  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.676515  8893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2764) took 1522 ms. Trace:
I20260709 13:02:39.676539  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.676992  8893 rpcz_store.cc:276] 0709 13:02:38.154233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.164375 (+ 10142us) service_pool.cc:224] Handling call
0709 13:02:38.184075 (+ 19700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.201765 (+ 17690us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:38.201811 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:38.202490 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.278312 (+ 75822us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:38.278357 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.278419 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.576124 (+1297705us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:39.648401 (+ 72277us) raft_consensus.cc:1685] finished
0709 13:02:39.648432 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.676437 (+ 28005us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17229,"raft.queue_time_us":21669,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":84,"spinlock_wait_cycles":24032512,"thread_start_us":74864,"threads_started":1,"wal-append.queue_time_us":297197}
W20260709 13:02:39.679009  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 246955us
I20260709 13:02:39.679422  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.679988  8820 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:39.720927  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:39.720945  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 206456us
I20260709 13:02:39.723666  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:39.678357  8816 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:39.729501  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:39.730607  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.731422  8816 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:39.725992  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.725071  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.733846  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:39.704563 15796 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.733569  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.743852  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2496) took 1680 ms. Trace:
I20260709 13:02:39.744566  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:39.727114  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 277997us
W20260709 13:02:39.744462  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 200626us
I20260709 13:02:39.727607 15633 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:39.766250  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:39.739265 15411 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:39.740113 15772 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.732932 15626 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.767244  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.769832  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.774346  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.772318  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:39.777428  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.745240 15503 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:39.747957  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:39.777721  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.778795  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.777498  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.784050  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.784621  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:39.778112  8895 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:39.785861  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.743660 15800 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:39.782549  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:39.788892  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:39.789379  8896 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:39.740572 15816 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:39.774497  8894 raft_consensus.cc:2376] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19. Re-sending same reply.
I20260709 13:02:39.745930 15820 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:39.792995  8892 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:39.794221  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.795171  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.768252 15633 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:39.800050  8816 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:39.800323  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.803599 15115 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:39.803973 15814 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:39.804723 15829 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.805338 15829 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:39.807729 15633 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.809074 15633 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.811218  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.812345  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.813607  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.817790  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.818224  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.818938  9445 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:39.819402  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.820606 15626 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:39.820852  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.821166 15626 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.821605  9446 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:39.823381 15626 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.824211 15808 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.825389 15808 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:39.825554 15506 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:39.825775 15808 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.826165 15808 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:39.782680 15503 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:39.827864 15503 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.829514 15824 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:39.830104 15824 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.832202 15824 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.783085 15813 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:02:39.837694  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.835036  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:39.856278  9447 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:39.864270  9449 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:39.859536  9441 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:39.864981  9446 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:39.956382  8893 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:39.957607  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.958217  8893 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:39.959231  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:39.959810  8893 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:39.960144  8893 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:39.961277  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.962893  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.964486  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:39.965183  9447 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:39.973767  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:39.975787  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 224146us
I20260709 13:02:39.977560  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.985240  8896 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:39.798112 15820 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.023878  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46082.3us
I20260709 13:02:39.796118 15816 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:39.989895  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.997272  9447 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
W20260709 13:02:40.021098 15792 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.962s	user 0.011s	sys 0.000s
I20260709 13:02:39.791301 15800 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:39.978102  8897 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:39.832799 15824 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:39.807684 15829 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:39.837554 15115 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:39.836299 15563 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.034570  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:39.784268 15772 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:39.973681 15496 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:39.979221 15626 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:02:40.064642  9440 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:40.032862 15816 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:40.030295 15820 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:40.053248  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:39.969974 15428 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:02:40.069924  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.039776 15827 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:40.048141 15800 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:40.054317 15812 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:40.080474  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2515) took 1056 ms. Trace:
W20260709 13:02:40.082896 15766 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.469s	user 0.011s	sys 0.017s
I20260709 13:02:40.046731 15824 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.030483  8893 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.106338  8891 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:40.097141  8822 rpcz_store.cc:276] 0709 13:02:39.024302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.026598 (+  2296us) service_pool.cc:224] Handling call
0709 13:02:39.026918 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.027275 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.027302 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.027858 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.058237 (+ 30379us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:39.058859 (+   622us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.059080 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.059157 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.016944 (+957787us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:40.063936 (+ 46992us) raft_consensus.cc:1685] finished
0709 13:02:40.080242 (+ 16306us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.080414 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":157,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":5560,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":179,"spinlock_wait_cycles":5958528,"thread_start_us":30011,"threads_started":1,"wal-append.queue_time_us":30446}
I20260709 13:02:40.123298  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:40.127578  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.095352 15115 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:40.173610  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:40.106652 15824 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.181015  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2955) took 2153 ms. Trace:
I20260709 13:02:40.183127  8890 rpcz_store.cc:276] 0709 13:02:38.027987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.036237 (+  8250us) service_pool.cc:224] Handling call
0709 13:02:38.112986 (+ 76749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.166161 (+ 53175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.050577 (+884416us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b540019d2d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:40.177783 (+1127206us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b540019d2d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:40.180861 (+  3078us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164559872}
I20260709 13:02:40.183943  9441 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:40.080426 15836 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:02:40.189751  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.186021  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.191327  8822 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:40.192371  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.193030  8822 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:40.198642  9442 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:40.202118  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.199652  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.118599 15656 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:02:40.211347  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.213322  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.215813  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.219511  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.099083 15827 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.229924  8890 raft_consensus.cc:2393] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:40.210518 15822 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.242452  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.198001 15808 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:40.246441  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 398211us
I20260709 13:02:40.247826  8824 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:02:40.250689  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 461961us
W20260709 13:02:40.251725  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2475) took 2419 ms (client timeout 2082 ms). Trace:
I20260709 13:02:40.233201 15825 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:40.252120  8815 rpcz_store.cc:269] 0709 13:02:37.827896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.829099 (+  1203us) service_pool.cc:224] Handling call
0709 13:02:40.247347 (+2418248us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400082ad4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:40.247449 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2715174656}
I20260709 13:02:40.258323  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.253189  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.259050  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.249014  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.244993  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.260324  8890 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:40.269416  9448 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:40.270175  8815 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:40.272646  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:40.280877  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.282652  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:40.244037 15503 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.263024 15836 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.289935  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 259227us
I20260709 13:02:40.274845 15280 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.245853 15822 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.261662 15825 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.299281 15503 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:40.299782 15825 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:40.300115 15503 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:40.300585 15503 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:40.301723  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.302300  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1209) took 1254 ms. Trace:
I20260709 13:02:40.304231  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.269742 15626 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:02:40.304101  8818 rpcz_store.cc:276] 0709 13:02:39.048190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.155533 (+107343us) service_pool.cc:224] Handling call
0709 13:02:39.155940 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:39.156375 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:39.304080 (+147705us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.841464 (+537384us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.841825 (+   361us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:39.956262 (+114437us) log.cc:844] Serialized 52 byte log entry
0709 13:02:39.956445 (+   183us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:39.956480 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.956543 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.302036 (+345493us) raft_consensus.cc:1685] finished
0709 13:02:40.302064 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.302227 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60738,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":31,"raft.queue_time_us":398489,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":9618,"spinlock_wait_cycles":6059904,"thread_start_us":450839,"threads_started":2,"wal-append.queue_time_us":269468}
W20260709 13:02:40.310983  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 56882.4us
I20260709 13:02:40.322722  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.303370 15836 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:40.328493 15633 consensus_meta.cc:287] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.001s
I20260709 13:02:40.328892 15844 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:40.329339 15844 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.329721  8890 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 14.
I20260709 13:02:40.270056 15115 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.329696 15844 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:40.306156 15280 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.329849 15827 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.326073 15496 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:02:40.328516 15836 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.377081  8892 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:40.387347 15748 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.387516 15280 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.388672 15808 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:40.369879 15843 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:40.346505 15838 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:40.338572 15832 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:02:40.391810  8895 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:40.391573 15824 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.393590 15824 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.387420  8892 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:40.395002  8892 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:02:40.372373 15503 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.339669 15772 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:40.382632 15822 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.393908 15838 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:40.406867 15838 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:40.407768 15838 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.409751 15820 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.412060 15820 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.412127  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.413172  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:39.745698  8823 rpcz_store.cc:276] 0709 13:02:38.063303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.133435 (+ 70132us) service_pool.cc:224] Handling call
0709 13:02:38.133772 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.133992 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:38.218928 (+ 84936us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.552371 (+1333443us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003913c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a
0709 13:02:39.738276 (+185905us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.738365 (+    89us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:39.738430 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.743676 (+  5246us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.743798 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153388,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":488,"raft.queue_time_us":213823,"spinlock_wait_cycles":37186304,"thread_start_us":311661,"threads_started":3,"wal-append.queue_time_us":288108}
I20260709 13:02:40.414021 15563 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:40.414459 15559 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:40.414652 15563 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.415033 15559 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.415107 15563 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.415592 15559 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:40.423549 15332 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.431346 15789 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.432005  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.433424  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:40.437621  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.443204  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.446280  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:40.448134  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 174415us
W20260709 13:02:40.449229  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 126611us
I20260709 13:02:40.448640  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
I20260709 13:02:40.450297  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:40.461833  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:40.450178  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:40.470731  8788 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.477674  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.477254  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:40.480567  9444 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:40.481815  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:40.475703  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 150323us
I20260709 13:02:40.419947 15844 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.500761  8789 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.460172 15740 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:40.500761  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:40.509110  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:40.511224  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 207074us
I20260709 13:02:40.480425 15664 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.504884  8788 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.481137 15633 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:40.480904 15824 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:40.513638  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 95934.9us
W20260709 13:02:40.520342  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.511957  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.523952  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.524726  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:40.527473  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.529188  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.504410 15820 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:40.491676 15850 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:40.507607 15822 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:40.538352  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.541829  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:40.511350 15626 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:40.480235 15816 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.484831 15800 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.513645 15633 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:40.532920 15595 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.546944  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.540884 15850 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.540270 15854 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.561956  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2505) took 1955 ms. Trace:
I20260709 13:02:40.540519 15822 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:40.554352 15633 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:40.570508  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.573820  8824 rpcz_store.cc:276] 0709 13:02:38.606262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.609856 (+  3594us) service_pool.cc:224] Handling call
0709 13:02:40.247536 (+1637680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.248162 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.560753 (+312591us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400082ad0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:40.561895 (+  1142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3766335360}
I20260709 13:02:40.534337 15428 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:02:40.553063 15852 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:40.513974 15559 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.579313  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:40.606160  8894 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:40.569188 15772 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.561573 15820 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:40.607878  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 103115us
I20260709 13:02:40.575300 15268 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.579276 15822 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.582083 15626 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:40.626008 15848 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:40.576874 15850 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:40.613181  8788 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:40.652647  8864 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:40.655952  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:40.664682  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:40.666332  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 150632us
I20260709 13:02:40.672335  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.673067  8891 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:40.674610  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 99599.5us
I20260709 13:02:40.677968  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:40.683545  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.627712 15824 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:40.571100 15595 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.686121  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.691617  8821 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 14.
W20260709 13:02:40.665346 15851 log.cc:927] Time spent T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:02:40.687228  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.611483 15804 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:40.690277  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:40.692859  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.686040  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:40.701658  9445 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 14.
I20260709 13:02:40.583014 15854 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.695827  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.697410  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:40.683740 15842 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.663757 15496 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:40.702008  9444 raft_consensus.cc:2376] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22. Re-sending same reply.
I20260709 13:02:40.700618  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.711513  8891 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:40.674769 15411 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.629012 15772 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.634032 15268 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:40.709969 15855 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:02:40.704528 15863 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.727051  8818 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:40.707870 15804 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:40.775383  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.708024 15869 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:40.727303 15836 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:40.760360 15862 log.cc:927] Time spent T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:02:40.759851 15859 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:40.719765 15860 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:40.732681 15844 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.763131 15865 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.800194 15858 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:40.801916 15877 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:40.803386 15860 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.809226 15865 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.811053 15871 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.811641 15871 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.727111 15842 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.806392 15650 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:40.772415 15428 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:02:40.791354 15268 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:40.724081 15411 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:40.828299  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.726133 15820 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.843470 15559 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.845083 15871 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.846076 15411 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.806614 15800 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.809235 15854 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.792845 15626 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.847388 15804 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:40.874279  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 192289us
I20260709 13:02:40.931135 15757 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:40.931680 15757 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:40.874382  8894 raft_consensus.cc:2468] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
W20260709 13:02:40.932682  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.933465  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:40.859318  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.891964 15871 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:40.891914 15559 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.839717 15563 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.837517 15650 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:40.925081  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.891227 15595 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.934301  8894 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:40.937824  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:40.817564 15816 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.949331 15868 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.949839 15868 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.939845 15872 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.952457 15872 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.954526 15872 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.925099 15772 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:40.779521 15863 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.971541 15863 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.972136 15757 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.948174  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:40.948980 15800 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.973455 15757 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:40.973592 15800 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:40.974227 15738 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:40.974664 15738 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:40.975764 15800 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.964577 15816 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:40.976480 15816 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.977178 15633 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.978824 15816 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.966248 15850 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.979228 15633 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.947826 15559 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:40.980863 15850 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:40.982616 15559 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:40.983584  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.984635 15664 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:40.985210 15664 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.987447  8823 raft_consensus.cc:1240] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:40.988898  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.989545 15411 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:40.990113 15411 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.990497  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:40.991991  8816 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:40.948176 15650 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.958652 15595 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:40.998322 15595 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.000690 15595 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.966543 15563 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:40.975421 15738 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.004359  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:40.943111 15824 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:41.005604  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:41.008963  8815 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:40.957211 15854 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:41.016125  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:40.960134 15872 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.990533 15411 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:41.022754  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 2964) took 1268 ms. Trace:
I20260709 13:02:41.004801 15595 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.001763 15826 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:41.025315  8889 rpcz_store.cc:276] 0709 13:02:39.754389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.754541 (+   152us) service_pool.cc:224] Handling call
0709 13:02:39.782432 (+ 27891us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.203318 (+420886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.203349 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.217404 (+ 14055us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.447650 (+230246us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:40.501047 (+ 53397us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.991544 (+490497us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b18003df9c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e23f3 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:41.000949 (+  9405us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b18003df9c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f335cc0a 00007f90fca9c219 00007f90f33e4a40 00007f90f33df4e4 00007f90f33df455 00007f90f9ec73c7 00007f90f64d99a7 00007f90f64e90bf 00007f90f64e9038
0709 13:02:41.001174 (+   225us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.001419 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.001504 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.022455 (+ 20951us) raft_consensus.cc:1685] finished
0709 13:02:41.022485 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.022669 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58658,"prepare.queue_time_us":54969,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":191730,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":54738,"spinlock_wait_cycles":24728320,"thread_start_us":579746,"threads_started":2,"wal-append.queue_time_us":275857}
I20260709 13:02:41.040374  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.006700 15633 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.947630 15871 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.017082 15804 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:41.052798  8713 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.024513 15828 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:41.034121 15738 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.019253 15620 consensus_queue.cc:1059] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 13:02:41.060635  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 364942us
I20260709 13:02:41.061342 15563 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:41.062523  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.067203  8712 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.069458  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.074163  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.024467 15772 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:41.076306  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.095258  8816 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:41.096024  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.096206  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.096704  8816 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:41.073953 15740 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:41.097236  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.097745 15620 raft_consensus.cc:3055] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:41.098105 15620 raft_consensus.cc:740] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.098382  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.098898 15620 consensus_queue.cc:260] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.099447  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.056777 15804 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:41.100072 15620 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.100351 15804 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.100869 15772 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.055675 15633 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.034314 15646 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:02:41.027457 15872 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.105845  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.106437  8823 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:02:41.114715  8823 pending_rounds.cc:85] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:02:41.114962  8823 pending_rounds.cc:107] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:02:41.115183  8823 raft_consensus.cc:2889] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.108295  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.110904  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.117569  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:41.118666 15883 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.119350 15646 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:41.119227 15883 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.119755 15646 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.120716 15646 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.121450 15883 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.121970 15883 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:41.123216 15883 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:41.124102  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 61698.8us
I20260709 13:02:41.124842  8713 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.126151 15559 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:41.125901 15822 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.127763 15563 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:41.128225 15563 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.129014 15563 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.129397 15633 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.128371 15822 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.131698 15872 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.132768 15559 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.133435 15559 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.135733 15879 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.136495 15879 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.136684 15824 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.138015 15850 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:41.135747  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.141937  8817 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.033967 15854 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.143172  8741 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:41.146231  8749 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:41.147545  8823 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:41.146165  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 468812us
I20260709 13:02:41.100442  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:41.151389  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.153213  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.153590  8824 raft_consensus.cc:2393] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:41.155798  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:41.158071  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.125943  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.156636  8743 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:41.170142  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58300 (request call id 1267) took 1041 ms. Trace:
I20260709 13:02:41.170879  8742 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:02:41.171968  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46100.9us
I20260709 13:02:41.173866  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.174558  8743 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:41.175060  8787 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:41.170671  9440 rpcz_store.cc:276] 0709 13:02:40.128568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.174482 (+ 45914us) service_pool.cc:224] Handling call
0709 13:02:40.174758 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.245189 (+ 70431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.245217 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.245258 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:40.245928 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.169906 (+923978us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54005f23d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:41.169954 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.170077 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161573120}
I20260709 13:02:41.127301 15738 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.177692  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.053319 15595 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.181813  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.181907  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.182686  8748 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.182840  8742 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:41.183900  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.184505  8745 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:41.074826 15628 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.097s
I20260709 13:02:41.188391  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.189071  8741 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
W20260709 13:02:41.170135 15849 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:02:41.188495  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.191255  8787 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:41.121965 15646 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.191497  8713 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.193457  8747 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:41.135823 15842 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.198431  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.199205  8744 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 21: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:41.200994  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.178365 15883 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:41.194738  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.207333  8747 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.052124 15826 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.202895  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.186553 15738 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.214404  8741 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:41.204452  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:41.205255  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.215839  8744 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:41.216761  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.217162  8787 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:41.211938  8817 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:41.212471  8742 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
W20260709 13:02:41.207172  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 163147us
I20260709 13:02:41.187897 15875 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:41.201866  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2774) took 1365 ms. Trace:
I20260709 13:02:41.221246  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.217315  8744 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:41.189391 15887 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:02:41.222481  8824 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:41.221182  8895 rpcz_store.cc:276] 0709 13:02:39.836298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.392483 (+556185us) service_pool.cc:224] Handling call
0709 13:02:40.392782 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.393162 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.393191 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.397427 (+  4236us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.502486 (+105059us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:41.201091 (+698605us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.201404 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.201508 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.201601 (+    93us) raft_consensus.cc:1685] finished
0709 13:02:41.201613 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.201798 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":498836,"prepare.queue_time_us":546273,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":937,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":11871,"thread_start_us":290226,"threads_started":2,"wal-append.queue_time_us":209768}
I20260709 13:02:41.243530  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.195269 15633 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:41.177405 15615 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:02:41.203205 15870 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.288514  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.206600 15626 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:41.290418  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50787.1us
W20260709 13:02:41.291905  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 65476.9us
I20260709 13:02:41.213487 15863 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:41.202768 15772 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.297561  8748 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:41.297783  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:41.261546  9414 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.510s (SENT) [suppressed 15 similar messages]
I20260709 13:02:41.299789  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.223738 15738 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.208478 15871 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.236721 15876 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:41.305179  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.250743  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.245590 15883 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:41.296500 15859 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:41.312158  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.252481 15559 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.314869 15869 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:41.339876  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 78378.6us
I20260709 13:02:41.319332 15821 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:02:41.319577 15882 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:41.250036 15664 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.324908 15854 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.252327 15633 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.332562  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.333590  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.354568  8748 raft_consensus.cc:2368] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
W20260709 13:02:41.354766  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.264576  8745 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:02:41.332620  8824 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:41.337895 15836 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:41.318692 15833 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:02:41.336935  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:41.355885  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.356490  8745 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:41.356633  8816 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:41.356804  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:41.341544  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 135916us
I20260709 13:02:41.344992 15821 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:41.359737  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.297154 15852 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:02:41.360838 15852 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:41.361199 15852 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.358911 15633 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.323942 15650 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.362006 15852 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.363188 15852 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.363325 15650 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.364013  8746 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:41.359513 15428 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:41.365700 15428 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.366134 15428 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.370642  8863 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.371353 15821 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.372326 15821 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.373658 15821 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.374078 15880 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:41.375892 15664 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.377704 15883 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.305367 15870 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.359063  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:41.359355  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.359855 15884 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.361383  8748 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:41.358808 15837 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:41.357815  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.385994  8744 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:41.342661 15869 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.397511  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:41.373832 15878 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:02:41.393975  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.401223 15650 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.403240  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.404145  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.411216  8816 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
W20260709 13:02:41.400616  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.404551  8741 raft_consensus.cc:2368] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:41.409361  8824 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:41.398576 15884 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.398414 15837 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.414556  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.399698 15836 raft_consensus.cc:2804] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:41.385643 15411 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.407047 15869 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.416214 15884 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.443331  8821 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:41.422583 15837 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.437968 15836 raft_consensus.cc:697] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.414032 15650 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.458184 15772 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:41.459167 15806 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:41.459589 15806 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.460161 15837 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.460932 15650 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.446297 15411 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.461576  8745 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:41.450380 15836 consensus_queue.cc:237] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.465430 15870 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.466606 15738 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.468039 15804 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.470005 15738 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.460538 15806 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.475486  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.476191  8740 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:41.482192 15826 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.484498  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.485232  8740 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.488401 15806 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:41.491204  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.492692 15905 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:41.494596  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.487883 15633 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.499869  8747 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:41.501935  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:40.662185  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:41.511698  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.444638 15563 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.535081  8715 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.515475 15332 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.528167 15595 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.530763 15633 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:41.543337  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.561457 15563 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.562613  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.564345  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.539758 15650 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:41.565577 15650 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.566038 15650 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.567883 15865 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.541258 15863 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:41.541764 15883 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:41.586432  9448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1176) took 1783 ms. Trace:
I20260709 13:02:41.543797 15880 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:41.524245 15428 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.544843  8863 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.593008  8893 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:41.593732  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48994.1us
I20260709 13:02:41.561254 15804 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.531352 15837 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.565033  8820 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:41.599143  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.600286  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:41.594498  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.577589  8741 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:41.586580  8894 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:41.541191  8748 raft_consensus.cc:1240] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:41.602727  8740 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:41.605054  8894 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:02:41.606215 15740 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.606308 15837 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:41.606752 15740 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:41.607188 15740 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.608773  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.605958  8820 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:41.611456  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:41.612079  8894 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.603281  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:41.591946  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.587016  9448 rpcz_store.cc:276] 0709 13:02:39.803570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.955679 (+152109us) service_pool.cc:224] Handling call
0709 13:02:39.956040 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.270062 (+314022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.586121 (+1316059us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540067cfd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:41.586360 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707808128}
I20260709 13:02:41.613078  8748 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.597381  8895 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:41.614148  9412 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.592828 15883 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.585578 15825 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:41.619876 15650 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.623076 15883 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:41.620563  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 493074us
I20260709 13:02:41.623292 15825 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.624568  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:41.593024 15880 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.609689 15844 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:41.625547 15825 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.625751 15844 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.622560 15428 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.595163 15838 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.627799  8712 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.629400  8894 raft_consensus.cc:2468] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:41.632270  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.632079 15844 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.633137 15563 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:41.628221 15826 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:41.634768  8890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40708 (request call id 1231) took 1414 ms. Trace:
I20260709 13:02:41.635948  8890 rpcz_store.cc:276] 0709 13:02:40.220005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.287489 (+ 67484us) service_pool.cc:224] Handling call
0709 13:02:41.634582 (+1347093us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b540015f754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:41.634686 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56487680}
I20260709 13:02:41.635064 15826 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:41.637771 15826 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:41.644093 15650 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.634327  8712 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.630023 15863 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.647879 15863 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.648280 15863 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.649489 15563 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:41.648072 15826 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.653573  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.660259 15826 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.663136 15563 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:41.664556 15563 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:41.663209 15650 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:41.665711 15563 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.669564  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:41.676925 15650 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:41.677621 15650 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:41.678156 15650 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:41.683498  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.684015  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.685840  8748 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:41.687862  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2498) took 3624 ms. Trace:
I20260709 13:02:41.688311  8819 rpcz_store.cc:276] 0709 13:02:38.063909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.142123 (+ 78214us) service_pool.cc:224] Handling call
0709 13:02:38.145213 (+  3090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.241926 (+ 96713us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:38.373879 (+131953us) log.cc:844] Serialized 14 byte log entry
0709 13:02:38.914056 (+540177us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b18004a11c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f23a 00007f90f64d9a2a
0709 13:02:38.937498 (+ 23442us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.937570 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:38.937631 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.194304 (+1256673us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b18004a11c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33b02a6 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419
0709 13:02:41.687639 (+1493335us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b540001fc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:41.687694 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.687809 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":174532,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":1811,"raft.queue_time_us":116,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5846,"spinlock_wait_cycles":251021184,"thread_start_us":115032,"threads_started":2,"wal-append.queue_time_us":592970,"wal-append.run_cpu_time_us":1294,"wal-append.run_wall_time_us":1020492}
I20260709 13:02:41.687516 15653 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:41.698338  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2538) took 1011 ms. Trace:
W20260709 13:02:41.702073  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 88002.2us
I20260709 13:02:41.701901  8818 rpcz_store.cc:276] 0709 13:02:40.687090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.692622 (+  5532us) service_pool.cc:224] Handling call
0709 13:02:41.698091 (+1005469us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54003e3fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:41.698261 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58047360}
I20260709 13:02:41.683512  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.699275  8822 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:02:41.708488  8893 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:41.711611  8712 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.713251  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:41.723404  8893 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50482 (request call id 2776) took 1873 ms (client timeout 1831 ms). Trace:
W20260709 13:02:41.725708  8893 rpcz_store.cc:269] 0709 13:02:39.836820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.958993 (+122173us) service_pool.cc:224] Handling call
0709 13:02:41.553742 (+1594749us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b58008c54d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d327c 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c
0709 13:02:41.709968 (+156226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425544064}
I20260709 13:02:41.723387  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.720829 15595 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.742283  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:41.750742  8743 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:41.721195 15901 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:41.743448 15871 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:02:41.787881  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 46267.2us
I20260709 13:02:41.820387 15620 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:02:41.821022  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:41.822057  9412 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:41.822754 15620 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:41.766498  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.824280  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.824486  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:41.824846  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58461.1us
I20260709 13:02:41.825021 15836 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:41.706570 15800 raft_consensus.cc:2764] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:02:41.747673 15863 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.704912 15653 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.826254 15836 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.827483 15863 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.827257 15836 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.827872 15653 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.799719  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.828693  8749 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.763214 15900 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.787693 15503 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:41.802186  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.830413 15900 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.830631 15503 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.830919 15900 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:41.831640 15503 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:41.804401  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 62166.7us
W20260709 13:02:41.753836 15909 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:02:41.750012 15884 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.742797 15595 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.820897 15642 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:41.822868 14821 raft_consensus.cc:2804] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:41.834790 14821 raft_consensus.cc:697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.834901 15818 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:41.835362 15818 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.835814 14821 consensus_queue.cc:237] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.797092 15865 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:41.706151 15804 raft_consensus.cc:2804] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:41.836376 15818 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.838091 15804 raft_consensus.cc:697] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:41.838581 15818 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.839295 15804 consensus_queue.cc:237] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:41.813964  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.823566 15620 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.845013 15620 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:41.850486  8744 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:41.855592  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.856803  9444 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.855208  8747 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:41.866122  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.867858  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.868115  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.869549  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.871434  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.873179  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.873765  9446 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:41.876947 15823 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:41.878268 15845 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:41.878680 15845 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:41.879585 15845 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.881186 15845 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:41.884354  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:41.888178  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.888422  9448 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:41.894565  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:41.897250  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 63160.1us
I20260709 13:02:41.789610 15892 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:02:41.900832  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.904235  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.908814  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.913070  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.913362  9444 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:41.914866  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.915177  9441 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:41.916386  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.918232  8747 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:41.931648  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.929885  8863 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.935390  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.896804 15563 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:41.937356  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:41.939929  9445 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.941697  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.942986  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:41.938799  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.950635  8746 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:41.819846 15650 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:41.998139  8789 leader_election.cc:400] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:41.995304  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:41.998739  8789 leader_election.cc:403] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.999065  8821 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.002898  9445 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:41.961934 15595 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:41.965888 15919 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:02:41.970314 15896 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:41.971774  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 672743us
I20260709 13:02:41.975020 15626 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:41.937886 15825 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:41.987478 15622 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:42.009562 15622 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.012024 15622 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:41.989764  8745 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:41.990540  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:41.993459 15599 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:41.993579 15879 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.995276 15806 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:42.009055 15825 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:42.013660 15806 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:42.014621 15825 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.014966 15806 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.016734 15806 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:41.996126 15818 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:41.996560 15868 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:41.997792  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:42.018170 15653 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:41.960772 15883 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.018584 15653 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:42.019567 15653 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.020586 15883 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.021442 15653 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:41.998271  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.024260  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.025347 15868 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.025743 15868 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.026127 15868 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:41.958475 15772 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:42.013417 15599 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.027997 15772 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:42.029583 15599 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:41.995870  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.030301  8822 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:42.031915  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.033156  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.036509  8821 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:42.037513  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:42.032716  9448 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
W20260709 13:02:42.041774  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43778.2us
I20260709 13:02:42.042078  8819 raft_consensus.cc:2393] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:42.040831  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:41.925879 15870 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:02:42.045833  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:42.046015  8787 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.042765  8789 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:42.042881  9448 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:42.047250  8787 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.061221  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.057817  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.049005  8743 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:42.056537  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.057579  9443 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:42.056229  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.055475  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:42.047992 15869 consensus_meta.cc:287] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.596s	user 0.005s	sys 0.009s
I20260709 13:02:42.083215  8864 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:42.071921  8788 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.033411 15896 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.092917  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45752.8us
I20260709 13:02:41.965137 15880 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.051769  8747 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:42.095233  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:42.006764 15646 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.085s
I20260709 13:02:42.046386 15875 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:42.115082  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:42.092677 15914 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:02:42.115274  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72583.9us
I20260709 13:02:42.045344 15883 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:42.130823  8789 leader_election.cc:400] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19.
I20260709 13:02:42.131971  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.134083  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:42.135144  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.135339  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:42.136475  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.137789  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.091993 15822 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.121639  9443 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:42.128289  8742 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:42.146235  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.151436  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:42.152726  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:42.154235  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82409.5us
I20260709 13:02:42.156020  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.156368  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:42.156553  8823 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.157581  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:42.157931  8818 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:42.161232  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.162318  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.061107 15882 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 13:02:42.162405  9443 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:42.112795 15837 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:42.112980 15832 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.108409 15896 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.168270  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.168762  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.188251  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.171958  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.189085  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.181304  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.189999  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.188875  8892 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:42.115674 15860 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.177167  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:42.192435  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:42.192730  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60984.6us
I20260709 13:02:42.190228  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:42.191259  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 92209.6us
I20260709 13:02:42.195230  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:42.123224 15892 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.201979  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.204277  8891 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:42.196820  8747 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:42.197100  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.198056  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.198328  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.153717 15899 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:42.158644 15912 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:42.199832  8713 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.206348  8889 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:42.193041  8896 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
W20260709 13:02:42.209353  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.180299 15872 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:42.110491 15869 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.186992 15883 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:42.210542  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.231065  8749 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:42.238804  8743 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:42.212123  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.182317 15833 raft_consensus.cc:3055] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:42.182929 15837 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.209533  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.189549 15819 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:02:42.235948  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 72013.6us
I20260709 13:02:42.239603  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:42.212834 15920 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:02:42.241549  8817 raft_consensus.cc:3060] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:42.244072  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44329.2us
I20260709 13:02:42.242303  8823 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:42.196580 15865 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:42.201929 15824 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.203603 15860 raft_consensus.cc:2764] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:42.272500  8894 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:42.275936  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.277391  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:42.277480  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 100867us
I20260709 13:02:42.279098  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.279321  8892 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
W20260709 13:02:42.283313  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.272290  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.239027 15620 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.243422 15837 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.240911 15833 raft_consensus.cc:740] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:42.268106 15563 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.229676 15892 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.270148 15865 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:42.282020 15804 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:42.285509 15646 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:42.339671  9442 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 15: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:42.309175  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 72945.7us
W20260709 13:02:42.311091 15924 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:42.311721 15892 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.313143 15865 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.304773 15563 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:42.314028 15883 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.289019 15827 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:42.323356 15926 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:42.287390 15650 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.331609 15821 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:42.335995  8746 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:42.348886  8817 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:42.351375  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 72299.8us
I20260709 13:02:42.299391 15332 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:42.364851  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:42.364764  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 262595us
I20260709 13:02:42.359423  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:42.376086  8822 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:42.285450 15854 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 13:02:42.384871  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.385522  8821 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:42.377722  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:42.377274  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 183040us
I20260709 13:02:42.367172 15563 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:42.367413 15927 log.cc:927] Time spent T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:42.399444  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:42.379953  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 133840us
I20260709 13:02:42.303476 15800 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 13:02:42.403430  8788 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:42.380542  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.391299  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.407328  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.377238 15869 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.415055  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:42.412348  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.420783  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.426172  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.406011 15615 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:42.381568 15821 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:42.362380 15871 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
W20260709 13:02:42.397023 15740 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.789s	user 0.013s	sys 0.000s
I20260709 13:02:42.428490  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.440937  8822 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:42.395946 15804 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.431143  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.434383  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.443346  8891 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:42.405939 15506 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.446990  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.438973  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.447635  8893 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:42.406243 15268 consensus_queue.cc:1059] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:02:42.451267 15871 consensus_queue.cc:1243] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:02:42.444681  8712 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.452549  8822 raft_consensus.cc:3055] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:42.452932  8822 raft_consensus.cc:740] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:42.427207 15923 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:42.402853 15646 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:42.414857 15892 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:42.415998 15650 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:42.446451  8890 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:02:42.447789  8895 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.443401  8894 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.437171 15615 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:42.419114 15642 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:02:42.447403 15506 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:42.453665  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.453943  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:42.454213  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.379618 15833 consensus_queue.cc:260] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.473030  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.446749  8864 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:42.432284 15860 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:42.429387 15880 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.409457 15835 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:02:42.442468 15900 consensus_meta.cc:287] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.611s	user 0.000s	sys 0.011s
I20260709 13:02:42.454835  8822 consensus_queue.cc:260] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.461555  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.455364 15888 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:42.464329  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.465595  8893 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:42.444979  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.452333 15332 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:42.480176  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:42.487218  8889 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:42.441771 15622 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:42.488574  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.489373 15900 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.489832  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.490959  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.491649  8744 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:42.485078  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.484853 15923 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.492449  8897 raft_consensus.cc:2368] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:42.493255  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.493777  8897 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.496369 15854 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.497044 15835 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.496915 15854 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.497551 15835 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.498299  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.499035  8893 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:42.485306 15850 raft_consensus.cc:2804] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:42.500934 15868 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.503206 15868 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.503698 15821 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.504922 15821 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:42.478749 15820 consensus_queue.cc:1059] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 13:02:42.506611 15917 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.697s	user 0.010s	sys 0.000s
I20260709 13:02:42.482930 15650 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:42.478106 15932 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:02:42.501232 15850 raft_consensus.cc:697] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:42.510533 15850 consensus_queue.cc:237] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.511241  9440 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:42.511649 15833 raft_consensus.cc:3060] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.513864 15884 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.514515  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.515746  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.516664 15880 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:42.517023 15880 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:42.517421 15880 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:42.518057 15888 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:42.518710 15888 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:42.520282  8889 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:02:42.520632  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.521646  8889 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:42.522815 15806 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.523334 15806 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.526499 15806 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.484258 15892 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:42.527084 15892 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:42.527590  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.437954 15937 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:02:42.527913 15892 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.528671 15937 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:42.529055 15937 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:42.529546 15806 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:42.530153 15892 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.529942 15937 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.531273 15937 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.533164  8897 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:42.449440 15853 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:02:42.534886  8896 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:42.535179  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:42.481125  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.452593 15804 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.466111 15620 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.536908  8890 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:42.447574 15740 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.472193 15506 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.488584 15615 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.551239  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.554382  9446 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:42.560863  8897 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:42.554544  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:42.483374  8894 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:02:42.562055  9440 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:42.564216  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:42.566012  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.480854 15856 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:42.514559 15854 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:42.576547  8787 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.579205  8787 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.566784  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:42.563862 15595 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:42.588284 15740 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.586508 15832 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:42.598868 15595 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:42.599915 15595 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.574386 15856 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:42.601807  8895 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:42.602277 15856 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.605655  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.606262  8897 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:42.607107  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.607895  8897 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 15: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:42.587447  8890 raft_consensus.cc:1240] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:02:42.610432  8787 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:42.585204  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:42.565625 15738 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.622121  8897 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:42.626824  8787 leader_election.cc:400] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Vote denied by peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) with higher term. Message: Invalid argument: T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17.
I20260709 13:02:42.628232  8787 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.633556  8890 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:02:42.597620 15845 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.003s
I20260709 13:02:42.638001  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:42.593538 15838 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:42.643142 15620 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.617038 15832 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:42.651543 15835 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:42.652252 15832 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:42.652837 15832 consensus_queue.cc:1243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:42.655727 15506 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.656905  8891 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:42.648205  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 132569us
I20260709 13:02:42.658003  8712 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.659736 15650 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:42.660261 15650 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:42.649848  8889 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:42.650959  8898 raft_consensus.cc:2410] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:02:42.651432 15620 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.661773 15896 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.662122 15620 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.661103 15650 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.662923  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.621116 15939 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.652119 15835 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:42.626276 15941 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:42.671572  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.593796 15824 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:02:42.701015  8815 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:42.692096  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.668022 15939 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.662721 15821 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.734244  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:42.735775  8712 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:42.666970 15838 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:42.698230 15860 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:42.745510  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 109916us
I20260709 13:02:42.711189 15642 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:42.731565 15939 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.746057 15860 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.747021  8787 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:42.730886 15883 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:42.759040  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 56512.4us
I20260709 13:02:42.765807  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:42.769691 15940 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:42.770133 15940 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:42.770501 15940 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:42.773483 15941 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:42.774657 15941 consensus_queue.cc:1059] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:42.777128  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:42.777792  8890 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:42.700299 15872 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:02:42.780098  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:42.742965 15800 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.782115  8890 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:42.700135 15923 consensus_queue.cc:1059] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 13:02:42.782739  8897 pending_rounds.cc:85] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 14
I20260709 13:02:42.787683  8897 pending_rounds.cc:107] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:02:42.788024  8897 raft_consensus.cc:2889] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:42.753952 15332 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:42.797848  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:42.741760 15832 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:42.797650  8819 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:42.732758 15949 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:02:42.768396 15860 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:42.711216 15880 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.739737 15821 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:42.908789 15772 consensus_meta.cc:287] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.880s	user 0.006s	sys 0.011s
W20260709 13:02:42.917557  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 168448us
W20260709 13:02:42.793730 15822 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.004s
I20260709 13:02:42.794085 15853 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:42.945670 15945 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:02:42.950232 15833 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:42.950836 15833 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.941542 15772 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:42.955547 15772 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.956122 15939 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.958894 15939 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:42.911731 15870 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:42.786798 15653 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.150s
I20260709 13:02:42.915979 15506 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:42.988112 15620 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.932000 15332 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:42.977463 15954 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:42.987075 15821 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:42.990299 15870 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:42.942366 15838 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:42.966277 15822 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:42.993525 15827 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.993786 15653 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:42.920656 15804 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:43.002822  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:43.005041  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:42.921226 15943 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.078s
W20260709 13:02:43.019265  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 235131us
I20260709 13:02:43.045316  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:43.045993  9446 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:43.046211  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:42.953081 15833 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:43.049005  8818 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.000288 15954 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:43.048394 15820 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:43.107398 15820 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.106516  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.048585 15599 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:43.016860 15860 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.105518  8818 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:43.112541  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:43.113157  8746 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:43.092881 15804 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 13:02:43.040516 15827 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.096904 15896 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:43.114655  9446 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:43.117676  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:43.049362 15615 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 13:02:43.119588  8788 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:43.111431 15653 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.111582 15820 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:43.129119  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:43.156358  8746 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
W20260709 13:02:43.197400  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 77861.7us
W20260709 13:02:43.128621 15937 consensus_meta.cc:287] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 13:02:43.122030 15939 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.155064 15837 consensus_meta.cc:287] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.826s	user 0.007s	sys 0.004s
W20260709 13:02:43.317628  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.125278 15943 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:43.126680 15826 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:43.117146 15599 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:43.262151 15892 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.003s
W20260709 13:02:43.261894 15563 consensus_meta.cc:287] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.849s	user 0.008s	sys 0.002s
I20260709 13:02:43.259455 15496 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:43.158780 15964 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:43.373608 15955 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 13:02:43.359397 15943 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.358758 15896 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:43.122578 15870 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.126853 15332 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:43.253088 15653 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.361945 15939 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:43.374387 15837 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.440778 15943 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:43.385558 15826 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.419181 15563 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.672425  8892 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:02:43.257781 15954 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:43.519685 15822 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:43.705780 15965 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.707739  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.709066  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.568044 15959 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:43.502929 15896 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.732829 15896 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:43.726477 15957 log.cc:927] Time spent T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.614s	user 0.000s	sys 0.001s
W20260709 13:02:43.756716 15968 log.cc:927] Time spent T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:02:43.763942  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:43.773756  8890 raft_consensus.cc:2393] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:43.459409 15964 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:43.776196  9413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.002s	sys 0.026s
I20260709 13:02:43.726864 15959 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:43.734560 15880 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.785156 15827 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:43.761991 15954 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.762501 15850 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:43.728081 15822 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.778688 15896 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:43.734247 15965 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:43.844527  8897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.064s	sys 0.192s
W20260709 13:02:43.852692  8897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.064s	sys 0.192s
I20260709 13:02:43.788529 15892 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.848191  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 331898us
I20260709 13:02:43.862522  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2792) took 1392 ms. Trace:
I20260709 13:02:43.864519  9446 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:43.854524 15940 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 1.084s	user 0.008s	sys 0.004s
I20260709 13:02:43.865067  8896 rpcz_store.cc:276] 0709 13:02:42.470197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.493725 (+ 23528us) service_pool.cc:224] Handling call
0709 13:02:42.534673 (+ 40948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.535289 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.862323 (+1327034us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b540015e350. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:43.862464 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93384576}
I20260709 13:02:43.854645 15804 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.854797 15967 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:43.856618 15496 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:43.830734 15850 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:43.882915 15822 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.854393 15896 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.892620  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:43.867242 15974 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:43.866498 15599 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:43.859230 15892 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.901051  8896 raft_consensus.cc:2393] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:43.874737 15959 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.905823 15959 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:43.913547 15850 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:43.914366 15850 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.914963 15850 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.890885 15860 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.930601  8896 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:43.934331 15850 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.933542 15822 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:43.930017 15563 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.938763  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:43.940034  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:43.877136 15940 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.941812  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:43.924482 15939 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:43.878589 15804 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.926903 15880 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:43.944540  9443 raft_consensus.cc:2368] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:43.928666 15837 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.930140 15650 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:43.937635 15961 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.879848 15967 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.937101 15826 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.956743  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:43.956923  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:43.957718  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.955940  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:43.960409  8713 leader_election.cc:400] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:43.963001  9440 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.965919  8713 leader_election.cc:403] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:02:43.965660  9448 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:43.975664  9440 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:43.923435 15974 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:43.937942 15822 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:43.957607 15838 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.959725 15332 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.979215 15985 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.001858 15965 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.002794 15896 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:43.963434 15650 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.015295 15940 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.021245 15882 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:44.024147 15974 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.026430 15967 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.028131 15972 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.030000 15748 raft_consensus.cc:3055] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:44.033704 15969 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:02:44.034355 15979 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.035799 15929 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 1.722s	user 0.010s	sys 0.028s
I20260709 13:02:44.036885 15804 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.038203 15895 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.038972  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2558) took 1883 ms. Trace:
I20260709 13:02:44.088557  8821 rpcz_store.cc:276] 0709 13:02:42.155268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.155393 (+   125us) service_pool.cc:224] Handling call
0709 13:02:42.385365 (+229972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.385891 (+   526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.038780 (+1652889us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54003e35d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.038915 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528227072}
I20260709 13:02:44.042186 15772 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:43.968360 15837 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.033119  8644 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 272) took 50238 ms (client timeout 15000 ms). Trace:
I20260709 13:02:43.977988 15865 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:43.960817 15980 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:44.048149  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.956259 15880 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.070576 15896 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:44.054544  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:44.076593  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.074251  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:43.959975 15496 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.085294  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2804) took 1310 ms. Trace:
I20260709 13:02:44.048859 15428 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:43.965541 15961 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:44.095659  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 137333us
W20260709 13:02:44.096086  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128657us
I20260709 13:02:44.053814  9445 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:44.078931 15748 raft_consensus.cc:740] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:44.088616 15820 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.825s	user 0.006s	sys 0.006s
I20260709 13:02:44.053341 15332 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.048915 15838 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.101986  8897 rpcz_store.cc:276] 0709 13:02:42.775270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.782075 (+  6805us) service_pool.cc:224] Handling call
0709 13:02:42.782500 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:43.039766 (+257266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.039809 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:43.041574 (+  1765us) log.cc:844] Serialized 52 byte log entry
0709 13:02:43.853138 (+811564us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:43.853180 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.853255 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.085033 (+231778us) raft_consensus.cc:1685] finished
0709 13:02:44.085055 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.085211 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22041,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":850188,"threads_started":2,"wal-append.queue_time_us":830853}
I20260709 13:02:44.100297  9442 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:44.175542  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.176452  8897 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:44.142614 15984 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:44.143281 15995 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:44.144125  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.144440  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.180862  8893 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:44.096174 15837 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.181946  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.151247  8745 raft_consensus.cc:3055] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:44.152354 15933 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:44.144148 15931 log.cc:927] Time spent T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.777s	user 0.022s	sys 0.011s
I20260709 13:02:44.055960 15985 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.177776  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.184780  8897 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:44.184813 15994 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.185823  8740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2494) took 2440 ms. Trace:
I20260709 13:02:44.102932 15892 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.182780  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.187023  8740 rpcz_store.cc:276] 0709 13:02:41.737847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.737989 (+   142us) service_pool.cc:224] Handling call
0709 13:02:41.738363 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.780745 (+ 42382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.780790 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.794758 (+ 13968us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.165484 (+370726us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.165522 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.165586 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.035151 (+1869565us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:44.163220 (+128069us) raft_consensus.cc:1685] finished
0709 13:02:44.163245 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.178201 (+ 14956us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3258,"raft.queue_time_us":42027,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7442,"spinlock_wait_cycles":100683648,"thread_start_us":269585,"threads_started":1,"wal-append.queue_time_us":501379}
W20260709 13:02:44.093070  8644 rpcz_store.cc:269] 0709 13:01:53.794891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.795375 (+   484us) service_pool.cc:224] Handling call
0709 13:02:44.033036 (+50237661us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:37.966889 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:38.046551 (+ 79662us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:38.046603 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:38.046620 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:38.047703 (+  1083us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:38.047800 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:38.047825 (+    25us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:02:38.048875 (+  1050us) tablet.cc:701] Row locks acquired
0709 13:02:38.048918 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:02:38.049081 (+   163us) write_op.cc:270] Start()
0709 13:02:38.049235 (+   154us) write_op.cc:276] Timestamp: P: 1783602158049015 usec, L: 0
0709 13:02:38.049268 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:02:38.146624 (+ 97356us) log.cc:844] Serialized 4456 byte log entry
0709 13:02:38.717014 (+570390us) op_driver.cc:464] REPLICATION: finished
0709 13:02:38.893379 (+176365us) write_op.cc:301] APPLY: starting
0709 13:02:38.893506 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:39.002761 (+109255us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:39.002906 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:39.019768 (+ 16862us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:39.020118 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:02:39.020178 (+    60us) write_op.cc:312] APPLY: finished
0709 13:02:39.020553 (+   375us) log.cc:844] Serialized 105 byte log entry
0709 13:02:39.029502 (+  8949us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:39.029635 (+   133us) write_op.cc:454] Released schema lock
0709 13:02:39.029705 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:39.041978 (+ 12273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:39.042010 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8591917,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":161483,"apply.run_cpu_time_us":17312,"apply.run_wall_time_us":156044,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50085,"num_ops":11,"prepare.queue_time_us":368398,"prepare.run_cpu_time_us":22209,"prepare.run_wall_time_us":378906,"raft.queue_time_us":270161,"raft.run_cpu_time_us":23761,"raft.run_wall_time_us":122507,"replication_time_us":629526,"thread_start_us":612223,"threads_started":4,"wal-append.queue_time_us":226104,"wal-append.run_cpu_time_us":60306,"wal-append.run_wall_time_us":1646378}]]}
I20260709 13:02:44.086136 15972 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.096504 15895 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.182804  8745 raft_consensus.cc:740] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:44.084681 15979 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.184909 15995 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.188197 15985 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.178719 15428 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.099081 15980 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.096288 15772 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:44.188916 15943 consensus_meta.cc:287] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.003s
I20260709 13:02:44.187858 15994 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.080803 15882 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.110431 15896 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.095192 15990 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 13:02:44.192204  8745 consensus_queue.cc:260] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.176930 15964 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.192909 15940 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.211571 15428 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.192751 15642 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:44.178459 15820 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.215317  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2555) took 3064 ms. Trace:
I20260709 13:02:44.217473  8816 rpcz_store.cc:276] 0709 13:02:41.150624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.454350 (+303726us) service_pool.cc:224] Handling call
0709 13:02:41.472995 (+ 18645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.987897 (+514902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.987931 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.988380 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.119818 (+131438us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.119861 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.120642 (+   781us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.143723 (+2023081us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b24000e9828. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f342109d 00007f90f605310e 00007f90f606a88c 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221
0709 13:02:44.209873 (+ 66150us) raft_consensus.cc:1685] finished
0709 13:02:44.209896 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.215245 (+  5349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":510053,"raft.queue_time_us":511104,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":31501696,"thread_start_us":131124,"threads_started":1,"wal-append.queue_time_us":307415}
I20260709 13:02:44.195667 15997 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.223171  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:44.223161 15838 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.216933 15642 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.205929 15943 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.220970 15940 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.191136 15984 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.249851 15961 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.237612 15650 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.252398  9445 raft_consensus.cc:1240] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:44.253054 15332 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.274793 15990 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.243780 15642 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:44.245007 15838 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.235849 15880 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.270942 15972 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:44.250332 15991 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:02:44.191105 15933 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.275640  8819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2565) took 2118 ms. Trace:
I20260709 13:02:44.230180 15748 consensus_queue.cc:260] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.286827  8819 rpcz_store.cc:276] 0709 13:02:42.157035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.238466 (+ 81431us) service_pool.cc:224] Handling call
0709 13:02:42.269445 (+ 30979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.910392 (+640947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.275421 (+1365029us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b540033b8d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.275587 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213909888}
I20260709 13:02:44.288887 15748 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:44.291262  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.291550 15984 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.293457 16002 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.296809 15979 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.299460 15653 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.299597 15943 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.302027  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.302651  8823 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:44.275345 15990 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.303131  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2580) took 1305 ms. Trace:
I20260709 13:02:44.305230  8823 rpcz_store.cc:276] 0709 13:02:42.997582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.301772 (+1304190us) service_pool.cc:224] Handling call
0709 13:02:44.303076 (+  1304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.235896 15997 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.306710 15994 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.311823  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:44.312827 15954 consensus_meta.cc:287] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 13:02:44.314414 15595 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.315330 15837 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.319761  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.287999 15980 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.312909  8821 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.322451  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.321504  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.325258  9441 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:44.326225  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:44.326931  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2579) took 1330 ms. Trace:
I20260709 13:02:44.327714  8823 rpcz_store.cc:276] 0709 13:02:42.996249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.325952 (+1329703us) service_pool.cc:224] Handling call
0709 13:02:44.326873 (+   921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.328805  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.329414  9442 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:44.330031  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.330334 15896 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.330662  8823 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:44.331614  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.281068 15838 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.332188  8823 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:44.295912 15882 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.331894  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.334585  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.304454 15895 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.337253  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:44.316444 16001 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:44.295230 16002 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.333321 15984 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.350370  8823 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:44.340633 15884 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:44.337859 15653 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:44.372200  8816 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:44.356613 15984 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.376912 15984 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:44.379886 15940 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.369803 15820 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.376001 15653 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.373255 15884 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:44.391279 15838 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.389605 15884 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:44.399514  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.387292 15995 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.403743  8821 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:44.404384  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 3111) took 1604 ms. Trace:
I20260709 13:02:44.405544  8821 rpcz_store.cc:276] 0709 13:02:42.800187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.311542 (+1511355us) service_pool.cc:224] Handling call
0709 13:02:44.404326 (+ 92784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.411675  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.413899 15653 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:44.404402 15819 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.067s	user 0.015s	sys 0.011s
W20260709 13:02:44.406136 15825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.375s	user 0.007s	sys 0.039s
I20260709 13:02:44.418540  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.420200 15990 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.421252  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.432928  9446 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:44.433423  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.433995  8897 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:44.434190  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.437733 15982 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.438791  8815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 3109) took 2406 ms (client timeout 1908 ms). Trace:
W20260709 13:02:44.441411  8815 rpcz_store.cc:269] 0709 13:02:42.032701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.032896 (+   195us) service_pool.cc:224] Handling call
0709 13:02:44.438600 (+2405704us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54002120d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.438727 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530912128}
W20260709 13:02:44.432819 15825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.402s	user 0.008s	sys 0.039s
W20260709 13:02:44.432081 15819 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.127s	user 0.016s	sys 0.012s
W20260709 13:02:44.443986 15876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.462s	user 0.013s	sys 0.018s
W20260709 13:02:44.445168 15876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.463s	user 0.013s	sys 0.018s
I20260709 13:02:44.449611  8895 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:44.450275 15933 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.450805  8895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1254) took 1848 ms. Trace:
I20260709 13:02:44.451162  8895 rpcz_store.cc:276] 0709 13:02:42.602909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.607054 (+  4145us) service_pool.cc:224] Handling call
0709 13:02:42.607302 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.450033 (+1842731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.450597 (+   564us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540046da54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.450753 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4237108608}
I20260709 13:02:44.442694  8821 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:44.452214  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.452454  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:44.452800  8895 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:44.443580 15595 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.443958 15997 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.454330  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.454792 15933 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.461658 15933 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.463047  8742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2496) took 2723 ms. Trace:
I20260709 13:02:44.463447  8742 rpcz_store.cc:276] 0709 13:02:41.739266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.831067 (+ 91801us) service_pool.cc:224] Handling call
0709 13:02:41.831403 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.161355 (+329952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.461122 (+2299767us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001c8254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.462980 (+  1858us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682359040}
I20260709 13:02:44.464011 15933 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.464649 15933 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:44.465004 15933 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.458745 16002 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.503036  9441 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:02:44.465257  8816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2574) took 1850 ms (client timeout 1898 ms). Trace:
I20260709 13:02:44.505098  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.510399  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.443729 15819 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:44.455221  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.511360  8895 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:44.480712 15876 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:44.511521  9442 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:44.474126 15653 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.509966  8816 rpcz_store.cc:269] 0709 13:02:42.615048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.222850 (+1607802us) service_pool.cc:224] Handling call
0709 13:02:44.465086 (+242236us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b5400082fd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.465200 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209129216}
I20260709 13:02:44.479902 15997 consensus_queue.cc:1059] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:44.466398  9446 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:02:44.455915 15595 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.513622  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.513824 15997 raft_consensus.cc:3055] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:44.514254 15997 raft_consensus.cc:740] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:44.455363  8894 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:44.482362  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.515440  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1239) took 1979 ms. Trace:
I20260709 13:02:44.502468  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.516153  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.454276  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.516986  8821 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:44.512301 15824 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:44.515046 15997 consensus_queue.cc:260] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.513002 15653 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.519493 15824 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:44.520015 15997 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.520484 15824 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.521220 15653 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.517063 15990 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:44.521806 15824 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.521881 15990 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:44.522471 15990 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.517082 15633 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:44.523386  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.460233 15865 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:44.459827  8897 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:44.453259 15642 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:44.524349  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.486538  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:44.526561  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.525942  8816 rpcz_store.cc:276] 0709 13:02:42.536309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.514673 (+1978364us) service_pool.cc:224] Handling call
0709 13:02:44.514949 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.515119 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.515147 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.515206 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:44.515239 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.515294 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.515386 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.528025  8821 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:44.530892  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2569) took 1918 ms. Trace:
I20260709 13:02:44.531198  8821 rpcz_store.cc:276] 0709 13:02:42.612894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.527482 (+1914588us) service_pool.cc:224] Handling call
0709 13:02:44.527740 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.530647 (+  2907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.530836 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.447088 15982 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.530236  8895 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:44.532692  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.533007  8897 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:44.533499  9444 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:44.534591  8816 raft_consensus.cc:1240] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:44.528077  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:44.535375  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.539991  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.443101 15984 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:44.541921 15930 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 2.228s	user 0.030s	sys 0.011s
I20260709 13:02:44.544049 15984 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.544902  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2549) took 3475 ms. Trace:
I20260709 13:02:44.545341  8824 rpcz_store.cc:276] 0709 13:02:41.069926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.439742 (+369816us) service_pool.cc:224] Handling call
0709 13:02:41.455744 (+ 16002us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.986429 (+530685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.986461 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.986962 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.271239 (+284277us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:42.271278 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.271341 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.453222 (+2181881us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:44.544610 (+ 91388us) raft_consensus.cc:1685] finished
0709 13:02:44.544636 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.544845 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530156,"raft.queue_time_us":531387,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":95537408,"thread_start_us":283899,"threads_started":1,"wal-append.queue_time_us":326376}
I20260709 13:02:44.547752  8824 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:44.548342 15748 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:44.548511 15738 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:44.548326  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2573) took 1933 ms. Trace:
I20260709 13:02:44.549821  8824 rpcz_store.cc:276] 0709 13:02:42.614801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.547120 (+1932319us) service_pool.cc:224] Handling call
0709 13:02:44.547515 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.548079 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.548273 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.551430  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2575) took 1847 ms. Trace:
I20260709 13:02:44.551733  8824 rpcz_store.cc:276] 0709 13:02:42.704033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.550611 (+1846578us) service_pool.cc:224] Handling call
0709 13:02:44.550918 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.551105 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.551135 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.551188 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:44.551223 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.551277 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.551380 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.548815 15748 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.553231 15748 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.553826 15832 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:44.536478  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2572) took 1921 ms. Trace:
I20260709 13:02:44.534713  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.524505 15933 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:44.557497  8821 rpcz_store.cc:276] 0709 13:02:42.614555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.532451 (+1917896us) service_pool.cc:224] Handling call
0709 13:02:44.535928 (+  3477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.536112 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.536138 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.536189 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:44.536218 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.536289 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.536414 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.559571  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2576) took 1800 ms. Trace:
I20260709 13:02:44.560748  8816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2571) took 1946 ms. Trace:
I20260709 13:02:44.563208  9445 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23.
I20260709 13:02:44.525368 15633 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.526674 15964 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.562479  8824 rpcz_store.cc:276] 0709 13:02:42.758827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.552610 (+1793783us) service_pool.cc:224] Handling call
0709 13:02:44.552889 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.558632 (+  5743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.558659 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.558703 (+    44us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:44.558738 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.558794 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.559519 (+   725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.563948  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2577) took 1778 ms. Trace:
I20260709 13:02:44.526226 15595 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.568259  8821 rpcz_store.cc:276] 0709 13:02:42.785406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.562789 (+1777383us) service_pool.cc:224] Handling call
0709 13:02:44.563288 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.563528 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.563564 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.563632 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:44.563679 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.563753 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.563878 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.564337  8816 rpcz_store.cc:276] 0709 13:02:42.614293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.530699 (+1916406us) service_pool.cc:224] Handling call
0709 13:02:44.534438 (+  3739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.535071 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.560694 (+ 25623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.569178  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 3112) took 1768 ms. Trace:
I20260709 13:02:44.571616  8824 rpcz_store.cc:276] 0709 13:02:42.800433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.568484 (+1768051us) service_pool.cc:224] Handling call
0709 13:02:44.568739 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.568890 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.568915 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.568960 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:44.568988 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.569035 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.569121 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.568883  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.573684  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.574124  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.574312  9449 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:44.567994 15738 consensus_queue.cc:1059] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:02:44.575722 15900 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.576042 15738 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:44.576448 15738 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:44.560112 15984 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:44.578325  8893 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:44.579154  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 3114) took 1778 ms. Trace:
I20260709 13:02:44.567596 15633 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:44.581218  8824 rpcz_store.cc:276] 0709 13:02:42.800893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.572795 (+1771902us) service_pool.cc:224] Handling call
0709 13:02:44.573279 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.573463 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.573488 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.573530 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:44.573564 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.573617 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.579111 (+  5494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.538901 15876 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:44.511554 15819 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.584113  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2578) took 1588 ms. Trace:
I20260709 13:02:44.577312 15738 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.585842  8824 rpcz_store.cc:276] 0709 13:02:42.995941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.583268 (+1587327us) service_pool.cc:224] Handling call
0709 13:02:44.583574 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.583768 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.583800 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.583882 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:44.583913 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.583964 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.584065 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.587584  8821 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:44.588106  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37570 (request call id 3115) took 1731 ms. Trace:
I20260709 13:02:44.588129  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.588456  8821 rpcz_store.cc:276] 0709 13:02:42.856618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.582649 (+1726031us) service_pool.cc:224] Handling call
0709 13:02:44.587437 (+  4788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.587942 (+   505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.588067 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.588882  9449 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:44.606048 16000 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.606747 16000 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.611783 15738 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.613271 14821 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:44.613708 14821 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:44.614542 14821 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.615950 15900 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:44.616469 15900 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.617792  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 3009) took 2162 ms. Trace:
I20260709 13:02:44.618232  8891 rpcz_store.cc:276] 0709 13:02:42.455925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.462516 (+  6591us) service_pool.cc:224] Handling call
0709 13:02:42.462795 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.657318 (+194523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.617528 (+1960210us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003bcc54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.617733 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446056192}
I20260709 13:02:44.620167 15900 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.571100 15964 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.622323 15964 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.584524 15876 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.583169 15984 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.630514  8821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1240) took 1630 ms. Trace:
I20260709 13:02:44.662881  8821 rpcz_store.cc:276] 0709 13:02:43.000539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.589113 (+1588574us) service_pool.cc:224] Handling call
0709 13:02:44.629865 (+ 40752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.630103 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.630139 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.630205 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:44.630249 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.630319 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.630443 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.581547 15990 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:44.628554  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.666096 15876 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:44.669898 15899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.363s	user 0.019s	sys 0.033s
W20260709 13:02:44.640421 16012 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:44.670499  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.705474  9443 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:44.673043  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.673013  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.673965  9449 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:02:44.672940  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.642035 15653 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.667279 15936 log.cc:927] Time spent T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 2.355s	user 0.021s	sys 0.011s
I20260709 13:02:44.676594  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.716648  9440 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 15.
I20260709 13:02:44.679505 15884 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.681412 15984 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.615513 14821 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.643754 16000 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.681186 15990 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.682462  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.691049 15933 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.596558 15748 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.724287 15748 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:44.725487  8744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2503) took 2838 ms (client timeout 1631 ms). Trace:
W20260709 13:02:44.725895  8744 rpcz_store.cc:269] 0709 13:02:41.883934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.884103 (+   169us) service_pool.cc:224] Handling call
0709 13:02:44.722045 (+2837942us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54001c7d54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.722176 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1395009792}
I20260709 13:02:44.718475 15653 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.725023  8891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a0644aef66444b58e8633da6993535c"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:02:44.727847  8891 raft_consensus.cc:606] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:44.728185  8891 raft_consensus.cc:614] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:44.728587 15653 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.729552 15642 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:44.733688  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.733556  8898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50492 (request call id 2988) took 2586 ms (client timeout 2524 ms). Trace:
W20260709 13:02:44.734261  8898 rpcz_store.cc:269] 0709 13:02:42.134723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.169394 (+ 34671us) service_pool.cc:224] Handling call
0709 13:02:44.721320 (+2551926us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000497d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.721475 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937954176}
I20260709 13:02:44.597252 15820 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.712060  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.715472  8818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37554 (request call id 2570) took 2102 ms (client timeout 1800 ms). Trace:
I20260709 13:02:44.719715  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.745038  9440 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:44.641381 15633 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.722024  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:44.744810  8818 rpcz_store.cc:269] 0709 13:02:42.613156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.045972 (+432816us) service_pool.cc:224] Handling call
0709 13:02:44.715300 (+1669328us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b54000811d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.715378 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5033856}
I20260709 13:02:44.746811  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:44.713470 15899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.399s	user 0.023s	sys 0.038s
I20260709 13:02:44.749831  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:44.787369  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.788089  8816 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:44.750860  8890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:02:44.789716  8890 raft_consensus.cc:606] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:02:44.753487  8894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2808) took 1010 ms. Trace:
I20260709 13:02:44.790987  8894 rpcz_store.cc:276] 0709 13:02:43.743456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.184806 (+441350us) service_pool.cc:224] Handling call
0709 13:02:44.185108 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.518985 (+333877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.753200 (+234215us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400247f54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.753424 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620844800}
I20260709 13:02:44.791699 15933 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.792958 15833 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:44.795141 15899 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:44.759507 15633 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.760092  8815 raft_consensus.cc:2368] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23.
I20260709 13:02:44.743526 15599 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.797366 15599 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.798992 15620 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.799678 15599 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.800968 15900 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.801520 15900 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.805755  8898 raft_consensus.cc:1240] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:44.806077  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.806527  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.806799  8893 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 17: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:44.807715  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.807757  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.808168  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.808648  8895 raft_consensus.cc:2410] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:44.808786  8896 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:44.784387  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:44.785583  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.805809  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.808876  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.809749  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.810048  8891 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.810315  8895 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:44.811481  8890 raft_consensus.cc:1217] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 17.15->[19.16-19.16]   Dedup: 19.16->[]
I20260709 13:02:44.811950  8890 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:02:44.812804  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.789026  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.813491  8894 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:44.814008  8816 raft_consensus.cc:2410] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:44.814688  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.815104  8816 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:44.815733  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.816143  8816 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:44.818686  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.820042  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.785622  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.821708  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:44.822546  8818 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:44.823009  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1242) took 1819 ms. Trace:
I20260709 13:02:44.823210  8818 rpcz_store.cc:276] 0709 13:02:43.004041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.822180 (+1818139us) service_pool.cc:224] Handling call
0709 13:02:44.822430 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.822848 (+   418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.822970 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.824334  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2581) took 1717 ms. Trace:
I20260709 13:02:44.824539  8818 rpcz_store.cc:276] 0709 13:02:43.107002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.823779 (+1716777us) service_pool.cc:224] Handling call
0709 13:02:44.823969 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.824088 (+   119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.824103 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.824133 (+    30us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:44.824149 (+    16us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.824180 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.824285 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:44.778800 15884 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:44.827924  9440 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:44.828225  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.828406  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2802) took 2253 ms. Trace:
I20260709 13:02:44.817410  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:44.828732  8889 rpcz_store.cc:276] 0709 13:02:42.575242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.575358 (+   116us) service_pool.cc:224] Handling call
0709 13:02:42.575629 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.661637 (+ 86008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.828192 (+2166555us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b540032b254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.828356 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170173824}
I20260709 13:02:44.830924  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:44.830334  8889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:02:44.833216  8889 raft_consensus.cc:606] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 29c7934ee528454785432732d01ed0a0
I20260709 13:02:44.833559  8889 raft_consensus.cc:614] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:44.835640  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:44.835779  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:44.836694  9440 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 15.
I20260709 13:02:44.832554 15836 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:02:44.838560 15821 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:44.843096  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:44.841904 15900 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:44.844699  8894 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:02:44.839016 15896 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:02:44.832599 15865 consensus_queue.cc:1059] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:02:44.839900  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1201) took 1795 ms. Trace:
I20260709 13:02:44.846999  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.847149  8891 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:44.838665  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.836738  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.848414  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:44.847446  9447 rpcz_store.cc:276] 0709 13:02:43.044637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.045217 (+   580us) service_pool.cc:224] Handling call
0709 13:02:43.045532 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.040930 (+995398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.040972 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.041020 (+    48us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:02:44.041050 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.839610 (+798560us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b5400740254. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:44.839681 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.839830 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289060608}
I20260709 13:02:44.849498  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:44.838268  8862 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:44.847740  8889 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 15.
I20260709 13:02:44.851709  8896 raft_consensus.cc:2410] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:02:44.854833 15942 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:44.855317 15942 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:44.855391  8820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1235) took 2510 ms. Trace:
I20260709 13:02:44.855783 15942 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.855902  8820 rpcz_store.cc:276] 0709 13:02:42.344397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.345172 (+   775us) service_pool.cc:224] Handling call
0709 13:02:42.345603 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:42.346008 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.346040 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:42.358412 (+ 12372us) log.cc:844] Serialized 52 byte log entry
0709 13:02:44.714073 (+2355661us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b1800387a60. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f60774bc 00007f90f33c548b 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1
0709 13:02:44.714239 (+   166us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:02:44.736581 (+ 22342us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.847460 (+110879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.848176 (+   716us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.848219 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:44.854767 (+  6548us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.854947 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22130,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":89841,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":15279,"spinlock_wait_cycles":2255744,"thread_start_us":176767,"threads_started":2,"wal-append.queue_time_us":164315,"wal-append.run_cpu_time_us":1597,"wal-append.run_wall_time_us":1001610}
I20260709 13:02:44.858309  8816 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:02:44.840782 15599 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:44.866731  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:44.858457 15411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.528s	user 0.034s	sys 0.044s
I20260709 13:02:44.876349  8894 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:44.865767 15900 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:44.825244 15820 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.881687 15900 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.879640 15599 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:44.883447 15599 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:44.883965 15820 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:44.880255 15411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.549s	user 0.035s	sys 0.045s
I20260709 13:02:44.885399 15899 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:44.886386 15411 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:44.886523  8817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 1238) took 2360 ms (client timeout 1619 ms). Trace:
I20260709 13:02:44.886786 15411 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:44.886986  8645 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 14, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 14 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:44.887625 15411 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:44.888677 15411 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:44.886960  8817 rpcz_store.cc:269] 0709 13:02:42.526223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.534928 (+  8705us) service_pool.cc:224] Handling call
0709 13:02:44.886328 (+2351400us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54003e30d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.886455 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5012499328}
I20260709 13:02:44.888739  8645 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:44.869279  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:44.893021  8645 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 14, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 14 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:44.898242  8823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37570 (request call id 3113) took 2097 ms. Trace:
I20260709 13:02:44.898609  8823 rpcz_store.cc:276] 0709 13:02:42.800665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.337046 (+1536381us) service_pool.cc:224] Handling call
0709 13:02:44.898023 (+560977us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b54003e2bd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:44.898185 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22112640}
I20260709 13:02:44.900777  8818 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:44.900228  8645 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 14, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 14 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:44.878130 15896 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:02:44.902139 15993 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:44.902604 15993 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:44.902969 15993 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:44.906464  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.907051 15833 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:44.907523 15833 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.907929 15833 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.844949 15982 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:44.908978  8864 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23.
I20260709 13:02:44.924700  8864 leader_election.cc:403] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.903800  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:44.907259  8645 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:44.972922  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.858072 15835 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:44.867913 15884 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:02:45.010841  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.024436  8822 raft_consensus.cc:3060] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:44.877993 14821 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.850946 15854 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:45.023833  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 120131us
I20260709 13:02:45.025619  8645 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 16, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 16 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:45.027920  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 296141us
I20260709 13:02:45.025969  8862 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:45.039917 15900 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.040666 15900 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:45.043720  8892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40708 (request call id 1250) took 2657 ms. Trace:
I20260709 13:02:45.044201  8892 rpcz_store.cc:276] 0709 13:02:42.385916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.387107 (+  1191us) service_pool.cc:224] Handling call
0709 13:02:42.387415 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.673609 (+1286194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.043415 (+1369806us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003723d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:45.043651 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2955108096}
I20260709 13:02:45.046471  8818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2582) took 1001 ms. Trace:
I20260709 13:02:45.046960  8818 rpcz_store.cc:276] 0709 13:02:44.045400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.831822 (+786422us) service_pool.cc:224] Handling call
0709 13:02:44.900448 (+ 68626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.901133 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.046236 (+145103us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b54003e2bd0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:45.046404 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156567424}
I20260709 13:02:45.051455  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.054281  8862 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.008985 15835 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.055999  8862 leader_election.cc:400] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23.
I20260709 13:02:45.007284 15823 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23.
I20260709 13:02:45.058565  9415 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:45.062474  8822 raft_consensus.cc:1275] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:02:45.059405  9412 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.062899  8822 pending_rounds.cc:85] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 15
I20260709 13:02:45.037956 15825 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:45.063247  8822 pending_rounds.cc:107] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:02:45.064011  8645 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 17. New cstate: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.066597  8822 raft_consensus.cc:2889] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:45.043094 15987 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:45.067938  8822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36246 (request call id 1237) took 2616 ms. Trace:
I20260709 13:02:45.048669 15772 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:45.068629  8822 rpcz_store.cc:276] 0709 13:02:42.452026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.452176 (+   150us) service_pool.cc:224] Handling call
0709 13:02:42.452437 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.024010 (+2571573us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5800462ed0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:02:45.067644 (+ 43634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.067881 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5186112896}
I20260709 13:02:45.041250 15900 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:45.113751  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.062350 15836 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:45.153612 15836 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:45.154292 15836 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:45.059885 15835 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.060369 16004 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:45.141180 15664 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:45.157784 16004 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.157847 15664 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.158419 16004 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:45.052814 15826 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
I20260709 13:02:45.065497 15825 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.162822 15825 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.158516 15664 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.041728 14821 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.167426 14821 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:45.062175 15962 consensus_queue.cc:1059] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:02:45.179530 15900 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.067787 15965 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.208990 15822 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.177062  8645 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 13, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 13 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.220443  8645 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 18, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 18 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:45.226100  8645 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 17, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.204521 15620 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:45.241387  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.241717  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.242522  8740 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:45.243877  8712 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.244092  8744 raft_consensus.cc:1275] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:02:45.244758  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 416226us
I20260709 13:02:45.247864  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.248235  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.250293  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.252081  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.257481  8740 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:45.260366  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.260977  8740 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.274132  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.281159  8740 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:45.287940  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.289480  8895 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:45.179766 15825 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.291162  8892 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:02:45.214247 15965 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.332279  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.334777  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.337268  8787 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.338423  8894 raft_consensus.cc:1240] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:02:45.239933 15949 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:45.314721  8740 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:45.317792  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.361372  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:45.320469  8741 raft_consensus.cc:1275] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:45.322804  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.326354  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.363168  8743 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.326625  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.327309  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.331748  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.373013  8898 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:45.373255  8645 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 14, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 14 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.377277  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.377923  8742 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.286979 15268 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:45.377787  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.389482  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.393380  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.392524  8741 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
W20260709 13:02:45.396607  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.397869  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.406137  8747 raft_consensus.cc:2393] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:45.382788 15876 consensus_meta.cc:287] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.675s	user 0.005s	sys 0.005s
I20260709 13:02:45.406966  8893 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:45.390343 15822 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:45.377491 15825 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.405258 15820 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:45.434005  8743 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:45.387352 15865 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.405534 15987 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:45.410663  8645 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 14, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 14 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:45.395076 15941 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:02:45.380620 15949 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:45.450829  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.433416 15599 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.004s
I20260709 13:02:45.417877 15876 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.443135  8745 raft_consensus.cc:3060] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.398679 15268 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.406881 15833 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.435039 16022 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.472951  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.458295 15884 raft_consensus.cc:993] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: : Instructing follower 29c7934ee528454785432732d01ed0a0 to start an election
W20260709 13:02:45.445214  8923 heartbeater.cc:646] Failed to heartbeat to 127.7.117.190:45111 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.117.190:45111 timed out after 15.000s (SENT)
I20260709 13:02:45.489837  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.492471  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.438241  8645 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 18, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:45.493227  8821 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19. Re-sending same reply.
I20260709 13:02:45.440660 16021 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:45.440814 15879 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.449568 16024 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.449818 16023 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:45.461974 15900 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.499490  8645 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 17, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.441615 15987 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.488375 15825 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:45.436200 15865 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.459079 15821 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:02:45.515514  8745 raft_consensus.cc:1275] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:02:45.516863  9440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.008s	sys 0.009s
I20260709 13:02:45.450654 15664 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.499060 15823 raft_consensus.cc:1081] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Signalling peer 29c7934ee528454785432732d01ed0a0 to start an election
I20260709 13:02:45.482838 15599 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.544436  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.545163  8743 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:45.545203  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:45.541213  9440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.008s	sys 0.009s
I20260709 13:02:45.545881  8816 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:45.541661  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45274 (request call id 1258) took 3165 ms. Trace:
I20260709 13:02:45.542187 15827 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:45.547801  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.549038  8645 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 13, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 13 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:45.550523  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.551070  8816 raft_consensus.cc:2376] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18. Re-sending same reply.
I20260709 13:02:45.550688  8645 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 18, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 18 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.546474  8745 rpcz_store.cc:276] 0709 13:02:42.376526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.376678 (+   152us) service_pool.cc:224] Handling call
0709 13:02:42.376992 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.151104 (+1774112us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400261c20. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f6446dc0 00007f90f64d11c0 00007f90f64d2231 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c
0709 13:02:45.042663 (+891559us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b18002f6b80. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f650a646 00007f90f644458e 00007f90f64c5f9e 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:02:45.442858 (+400195us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b580072c2d0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f642b0aa 00007f90f64ad447 00007f90f64c5fe7 00007f90f64c77de 00007f90f64d1b49 00007f90f64d2416 00007f90f64ced8f 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4
0709 13:02:45.540482 (+ 97624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.541591 (+  1109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682590720}
I20260709 13:02:45.553671  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.554263  8816 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.556857  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.557677  8822 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:45.566020  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.566753  8749 raft_consensus.cc:2468] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:45.568356  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.568962  8747 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:45.570487  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.571333  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.572117  8741 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 22: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:45.574958  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.575523 15833 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.575641  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:45.576664  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.609230  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.610762  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.611306  8893 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.576568  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.613168  8748 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.507470 15879 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.614580  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.615301  8741 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.577548  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.577654  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.578497  8645 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 12, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 12 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:45.623440  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.509759 16023 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.582636 15987 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:45.584221 15835 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.585009 15876 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.585883 15949 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.587146 15268 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.508450 16024 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.588358 15970 raft_consensus.cc:3055] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:45.590898 16004 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.592648 15615 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:45.499395 16022 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.540360 15825 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.593683 15827 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.593953  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:45.673976  9445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.023s	sys 0.038s
I20260709 13:02:45.595713 15664 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:45.674729  9445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.023s	sys 0.039s
I20260709 13:02:45.674664  8740 raft_consensus.cc:2393] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:45.596977  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.597803 15599 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:45.599700 15942 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.003s
W20260709 13:02:45.548410 15871 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 13:02:45.655259 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.028s	sys 0.054s
I20260709 13:02:45.606962 15836 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.624785  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.626077  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.692706  8821 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:45.693396  8744 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:02:45.626828  8748 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:45.635083  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.635222  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.694470  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.507156 16021 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.637490  8747 raft_consensus.cc:2376] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19. Re-sending same reply.
W20260709 13:02:45.689731 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.028s	sys 0.055s
I20260709 13:02:45.599982  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.644119  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:45.636010 15993 consensus_meta.cc:287] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.723s	user 0.005s	sys 0.005s
I20260709 13:02:45.701226  8820 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:02:45.642650  9415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.004s	sys 0.020s
I20260709 13:02:45.670877  8741 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:45.703622  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.707161  8897 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:45.707885  8893 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:45.690706 15982 raft_consensus.cc:3055] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:45.704934  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.710537  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:45.710592  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110723us
I20260709 13:02:45.673985 15615 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.670734  8645 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 18, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:45.675995  8745 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:45.678474  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.711674  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.688325  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.511849 15900 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.690415  8891 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:45.635818  8742 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:45.716604  8742 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:45.665717 15970 raft_consensus.cc:740] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:45.717432  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.697765  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:45.699339  9412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.018s	sys 0.046s
I20260709 13:02:45.696776  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:45.720810  9412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.019s	sys 0.046s
I20260709 13:02:45.720899  8741 raft_consensus.cc:2393] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:45.721182  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 661880us
I20260709 13:02:45.700235 15268 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:45.702387  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.720841  8743 raft_consensus.cc:2376] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17. Re-sending same reply.
I20260709 13:02:45.641883  8816 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
W20260709 13:02:45.714504  9415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.005s	sys 0.020s
W20260709 13:02:45.733632  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 675149us
I20260709 13:02:45.734064  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.734863  9415 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:45.690917 15942 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:45.677234 15899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.006s	sys 0.029s
I20260709 13:02:45.715119 15949 raft_consensus.cc:2764] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:02:45.753376  8645 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:45.761981  8712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.026s	sys 0.046s
W20260709 13:02:45.762552  8712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.026s	sys 0.046s
I20260709 13:02:45.735960  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:45.745875  8863 leader_election.cc:336] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.631s (SENT)
I20260709 13:02:45.721843 15994 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:45.752171  8749 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:45.722172  9412 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.709982 15982 raft_consensus.cc:740] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:45.765846 15982 consensus_queue.cc:260] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.767524 15982 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.710085 15993 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.732311 15595 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:45.771034  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 48924.4us
I20260709 13:02:45.747535 15987 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:45.758247  8645 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 4 to 15. New cstate: current_term: 15 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:45.773020  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.778453  8815 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:45.759213 15899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.007s	sys 0.029s
I20260709 13:02:45.776070  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:45.751338 15772 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:45.756976 15664 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:45.753001 15823 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:45.784291  8820 raft_consensus.cc:2468] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:45.756785 15268 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:45.790951  8743 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:45.803185  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.807181  9448 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:45.807796  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:45.777729 15943 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:45.774950 15977 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.818585  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:45.787082 14821 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.024s
I20260709 13:02:45.813549  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:45.813691  8645 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 16, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 16 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:45.825304  9441 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:02:45.777554 15884 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:45.822841  9446 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:45.831166  9442 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:45.828518  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:45.838266  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:45.802214 15959 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:45.840155  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:45.842542  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.843815 15835 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.844072  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.802068 15664 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:45.847155  8745 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:45.850020  8741 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.853135  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:45.853724  9448 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.803574  8815 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:45.826361  8645 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 16, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 16 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.780138 15987 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.854974  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:45.810668 15823 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.812129 15772 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:45.871825 15772 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.873502 15268 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.816757 15943 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.875155  9448 raft_consensus.cc:2468] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:45.875326 15943 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:45.852386 15664 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.861577  8815 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:45.824216 14821 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.000s	sys 0.024s
I20260709 13:02:45.884704 15664 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.889325  8815 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:45.823853 15876 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.806409 15642 consensus_queue.cc:1059] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 13:02:45.833496 16034 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.870810 15823 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.868417  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.900575  8743 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:45.852505 15827 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:45.829654 15996 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.866600 15987 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.867233 15825 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.929739  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:45.864706 15959 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:45.840297 15977 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:45.890465 15900 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.937126  9446 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
W20260709 13:02:45.940140  9447 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.017s	sys 0.044s
I20260709 13:02:45.940905  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:45.941684  9447 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.017s	sys 0.045s
I20260709 13:02:45.905195 16023 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.943775  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:45.805375 15970 consensus_queue.cc:260] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.955044  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:45.922989 16004 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.955941  8893 raft_consensus.cc:1240] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:02:45.958806  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.923300 15836 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.959420  8747 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:02:45.921221 15996 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:45.953564  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 301797us
W20260709 13:02:45.962621  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.964149  9440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1221) took 1019 ms. Trace:
I20260709 13:02:45.975289  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.959764  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:45.975625  9447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44010 (request call id 1220) took 1031 ms. Trace:
I20260709 13:02:45.976161  8741 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:45.968875 16021 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.968945 15996 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.969926 15827 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:45.972443 15865 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.905238 16034 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.964972 15615 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.006520 15615 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:46.007092 15615 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.007938 16034 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.008853 15664 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:46.012524  8819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36246 (request call id 1241) took 3009 ms (client timeout 1573 ms). Trace:
I20260709 13:02:46.014071 15959 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:45.955253 16033 raft_consensus.cc:3055] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:46.017575 16033 raft_consensus.cc:740] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:46.018687 16033 consensus_queue.cc:260] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.020022 16033 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.021550 15820 consensus_queue.cc:1059] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:46.022593 15820 raft_consensus.cc:3055] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:46.022938 15820 raft_consensus.cc:740] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:46.023785 15820 consensus_queue.cc:260] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.025866 15993 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.026625 15942 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.964808 16024 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.030938 16022 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:45.999737 15899 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:46.031980 15899 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:46.032538  8748 raft_consensus.cc:1240] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.033012 15899 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:46.012964  8819 rpcz_store.cc:269] 0709 13:02:43.003403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.290994 (+1287591us) service_pool.cc:224] Handling call
0709 13:02:46.012298 (+1721304us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003e26d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:46.012442 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3474667776}
I20260709 13:02:46.038138 15977 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.902989 15876 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.038762  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.040066  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.040834  9446 raft_consensus.cc:2393] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 20: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:46.041394 15827 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.044013 15827 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:45.976727  8787 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:45.975468  9440 rpcz_store.cc:276] 0709 13:02:44.944327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.945482 (+  1155us) service_pool.cc:224] Handling call
0709 13:02:44.945820 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.946044 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:45.552894 (+606850us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.963706 (+410812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.963774 (+    68us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:45.963839 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.963932 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.964073 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49907,"prepare.queue_time_us":798085,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3622,"raft.queue_time_us":57723,"thread_start_us":927731,"threads_started":2}
W20260709 13:02:46.047947  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 71303.4us
I20260709 13:02:46.048271  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:45.982336  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:45.976174  9447 rpcz_store.cc:276] 0709 13:02:44.943944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.944130 (+   186us) service_pool.cc:224] Handling call
0709 13:02:44.944567 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.945033 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.945074 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.007672 (+ 62598us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.293709 (+286037us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:45.945458 (+651749us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.975340 (+ 29882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.975415 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.975456 (+    41us) raft_consensus.cc:1685] finished
0709 13:02:45.975467 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.975573 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28941,"prepare.queue_time_us":514229,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1528,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":17095,"thread_start_us":806555,"threads_started":2,"wal-append.queue_time_us":397284}
I20260709 13:02:46.049892  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.050433  9447 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:46.051168  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.051674 15854 raft_consensus.cc:2804] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:46.052155  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.052893  9447 raft_consensus.cc:2393] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:46.046543 15615 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.025035 15820 raft_consensus.cc:3060] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.994339  8743 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.001222  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:45.985395  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.055423  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.049178  8889 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.055684  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.056053  8897 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:46.059859  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.060796  9446 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:46.061448  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.056941  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:46.058823  8823 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
 from {username='slave'} at 127.0.0.1:37554
I20260709 13:02:46.066671  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.070281  9447 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
I20260709 13:02:46.064368  8823 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.072546  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.074506  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:46.075004 15833 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:46.075392 15833 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.075769 15833 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.075821  9447 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:46.073527 15615 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:46.078928  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.040859 15876 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.087474  9440 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:46.079418  8823 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.053105 15996 consensus_queue.cc:1059] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:02:46.057586 15854 raft_consensus.cc:697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:46.121177  9446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.069494 15824 raft_consensus.cc:3055] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:46.133452 15854 consensus_queue.cc:237] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.141878  9442 raft_consensus.cc:1240] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.141945  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.143286  8743 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:46.058856 15943 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:46.147975  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.122336 15942 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:46.130586  8823 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:46.176723  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 403572us
I20260709 13:02:46.179059  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:46.100315 15633 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:02:46.177620  9446 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:46.127317 15824 raft_consensus.cc:740] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:46.184576  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.185456  8743 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 22: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:46.185485  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 413145us
I20260709 13:02:46.116287 15615 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.186713  9449 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:46.187805  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:46.198897  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.198848  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.219439  8864 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.179149 15942 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.189066 15506 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:46.208590  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.189370  8823 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.233093  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.233538  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.234895  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:46.240679  9442 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.248725  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.229859 15975 raft_consensus.cc:2804] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:46.229642 15826 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:46.227955 15884 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.239980 15595 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:46.229106 15942 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:46.269172  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.271088  8789 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.963s (SENT) [suppressed 36 similar messages]
I20260709 13:02:46.233158 15871 consensus_queue.cc:1059] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:02:46.286867  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.287861  8746 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.207808 15824 consensus_queue.cc:260] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.283056 16022 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:46.293025 16022 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:46.293485 16022 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.297658 15993 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.298269 15993 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.300320 15993 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:46.259248 15826 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:46.305588  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.263690 15975 raft_consensus.cc:697] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.312307  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:46.318917  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 240042us
I20260709 13:02:46.320855  8748 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.309824 15879 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:46.328436 16042 log.cc:927] Time spent T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:02:46.333317 16043 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:02:46.334168 16041 log.cc:927] Time spent T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:02:46.334578 15965 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.337569 15949 consensus_queue.cc:1059] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:46.338584 15865 raft_consensus.cc:3055] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:46.338943 15865 raft_consensus.cc:740] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.339707 15865 consensus_queue.cc:260] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.339888 15833 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.320674 15595 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.341012  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75829.9us
I20260709 13:02:46.320926 16024 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.342358  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.259521 15884 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.343387  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.355984  8741 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:46.347054  8712 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.357102  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.358196  8741 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:46.358296  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.359365 16036 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:46.359959 16036 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:46.360797  8712 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:46.361827 16050 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:46.363144 16050 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.365077  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.365912  8746 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:46.366453  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:46.367199  8897 raft_consensus.cc:3055] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:46.367542  8897 raft_consensus.cc:740] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.369484  8748 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:02:46.367161  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.370934  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.371743  8741 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:46.373842  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.374742  8748 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:46.320364 15943 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.375037  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.379314  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.375965  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.355826 15595 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.376761 16050 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:46.378135 16047 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:46.386685  9445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1377) took 1327 ms. Trace:
I20260709 13:02:46.387061  9445 rpcz_store.cc:276] 0709 13:02:45.058808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.058936 (+   128us) service_pool.cc:224] Handling call
0709 13:02:45.068581 (+  9645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.675352 (+606771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.675398 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.675992 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.138216 (+462224us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:46.138268 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.138359 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.386472 (+248113us) raft_consensus.cc:1685] finished
0709 13:02:46.386496 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.386628 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5113,"raft.queue_time_us":132447,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7236,"thread_start_us":1067618,"threads_started":2,"wal-append.queue_time_us":629952}
I20260709 13:02:46.388522  8890 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:46.305524 15826 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.390404 15899 consensus_queue.cc:1059] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:46.404462 15975 consensus_queue.cc:237] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.367887 15824 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:46.385849  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.573s (SENT)
I20260709 13:02:46.414938 15884 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:46.372704 15965 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.416002 15965 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.418745 15965 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.380764  8897 consensus_queue.cc:260] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.420627  8897 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.420499  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.434206 15595 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.435902 15833 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.436386 15615 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.368296 16024 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.446720 15970 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.454666 16024 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:46.459092 16022 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.460675 15942 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.421841 15987 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.471460 15987 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:46.469384 15942 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.475358 16022 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:46.476198 15994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.016s	sys 0.028s
W20260709 13:02:46.556740 15994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.016s	sys 0.028s
I20260709 13:02:46.511473  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:46.522918  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.559574  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.560521  8740 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:46.381199  8741 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:46.378456  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.525004  8743 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:46.529244  8746 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:46.537259  8862 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.541245  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.542420  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:46.543570  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.543949  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:46.544888  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:46.546859  8895 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:46.548683  8896 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:46.548529 16032 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:46.569622  8890 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.550483  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.570837 16032 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.572767  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:46.573732  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.574326  8894 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.574826  8746 raft_consensus.cc:2393] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.575004  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.575742  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.575899  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.576452  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:46.573679  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.578451 15506 raft_consensus.cc:2804] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:46.578639  8862 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:46.578877  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.578454  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.579370  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.579722 16073 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:46.580257  8891 raft_consensus.cc:2393] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.574340 16032 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:46.580827  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.544947  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.581833  8889 raft_consensus.cc:2393] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.564328 15994 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.584075  8817 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.585047  8818 raft_consensus.cc:1240] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:46.585382  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:46.585930  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.585991  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.586974  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:46.587704 15821 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:46.565059  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 53992.5us
I20260709 13:02:46.590081  8896 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:46.580308  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.588604 15823 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:46.594449  8823 raft_consensus.cc:2410] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:46.595731  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:46.598130  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.598084  8898 raft_consensus.cc:1240] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:02:46.586090  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:46.590575 15506 raft_consensus.cc:697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.602914  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.601627 15826 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:46.604815  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.606084  8822 raft_consensus.cc:2368] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
I20260709 13:02:46.609582  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:46.611387  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.611264 16032 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:02:46.613749  8889 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.605093 15615 consensus_queue.cc:1048] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:02:46.617419  8824 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:46.611274  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.618477  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.619968  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:46.620551  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.606928 15821 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.619798 15987 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:46.625984  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:46.626157 16024 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:46.628304 15595 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:46.628902 15595 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.630530 15595 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.631680  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.632115 16053 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:46.632608 16053 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.628408 16022 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:46.633513 16022 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.634245 16022 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:46.634107 16053 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.635259  8742 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:46.634270  8749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51680 (request call id 2533) took 1059 ms. Trace:
I20260709 13:02:46.636202 16053 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.633437  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.640395  8744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2538) took 1034 ms. Trace:
I20260709 13:02:46.631026  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.636719  8749 rpcz_store.cc:276] 0709 13:02:45.574367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.577345 (+  2978us) service_pool.cc:224] Handling call
0709 13:02:46.633840 (+1056495us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400021cd4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64d32fa 00007f90fb494b17 00007f90f4a0f6d4 00007f90f4a0f662 00007f90f4a0f591 00007f90f4a0f50c 00007f90f4a0e7c2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:46.634061 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297232256}
I20260709 13:02:46.642370  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.640561  8824 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.642583 15821 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.633131  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.640844  8744 rpcz_store.cc:276] 0709 13:02:45.594642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.595135 (+   493us) service_pool.cc:224] Handling call
0709 13:02:45.595433 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.693737 (+ 98304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.628797 (+935060us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b54000b5754. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:46.629100 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224505600}
I20260709 13:02:46.645844  8818 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:46.646858  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.648505  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.649852  8741 raft_consensus.cc:2410] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:46.651058  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.652495  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.655495  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.633090 15595 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.667563  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:46.668112  9414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.668740  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.617404 15994 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.671545  9444 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:46.674193  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.675623  8818 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:46.677686  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.681733  8744 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:46.693699  8746 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:46.701301 15506 consensus_queue.cc:237] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.702975 15975 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:46.703824 15975 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:46.700126  8748 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.709465  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.712215  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:46.713102 15865 raft_consensus.cc:3060] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.717132  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.718223  8818 raft_consensus.cc:2393] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 18: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:46.719470  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:46.720284  9449 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.705250 15975 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.725834  8823 raft_consensus.cc:1240] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.713414  9447 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:46.710515  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.745445  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.746392  8745 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:46.749696  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:46.749886  8897 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
W20260709 13:02:46.754122 16059 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.377s	user 0.004s	sys 0.001s
W20260709 13:02:46.756341  8789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.759776  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:46.769870  8894 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:46.770196  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.771282  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.744886 15982 raft_consensus.cc:3055] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:46.752908  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:46.774129  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 66152us
I20260709 13:02:46.784337  8744 raft_consensus.cc:3060] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:46.788041  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 1.042e+06us
I20260709 13:02:46.792200  8746 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:46.802219  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.802274 16025 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:02:46.805497 15615 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:46.806676 16025 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:46.806950  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 53738us
I20260709 13:02:46.807242 16025 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.809504  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.809773 15871 consensus_queue.cc:1059] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:46.811563 15871 raft_consensus.cc:3055] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:46.811947 15871 raft_consensus.cc:740] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:46.814832  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.811753  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.809744  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:46.813124 15871 consensus_queue.cc:260] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.818313 15615 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:46.817694  8740 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:46.821646 15871 raft_consensus.cc:3060] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.824636  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.813823 16025 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:46.830643 16025 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.831787  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:46.779914 15982 raft_consensus.cc:740] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:46.835773 15982 consensus_queue.cc:260] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:46.837404  8740 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:46.838090 15938 raft_consensus.cc:3055] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:46.838032 15615 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:46.859987  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.861058  8746 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:46.862253  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.863279 15826 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:46.863626  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.858143  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.865896  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.867126 15949 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:46.867550  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:46.868464 15949 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:46.863850 15826 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.836860  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.870327  8742 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:46.871387  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.872280  8742 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.878922  8748 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:46.825119  8747 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:46.917024  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.917209  9449 raft_consensus.cc:2468] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:46.837271 15982 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:46.928081  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.931077  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.933776  8740 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.935068  8890 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
W20260709 13:02:46.941708  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 72773.6us
I20260709 13:02:46.855398 15938 raft_consensus.cc:740] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:46.951752  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.922245 16032 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:46.952996  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.954849  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:46.959697  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:46.964534  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.965369  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.974157  8742 raft_consensus.cc:1240] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:46.975677  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:46.981637  8743 raft_consensus.cc:2410] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:46.956970  8746 raft_consensus.cc:2376] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20. Re-sending same reply.
W20260709 13:02:47.000140  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43869us
W20260709 13:02:47.001505  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.004312  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 216424us
I20260709 13:02:46.997234  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.016297  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.017035  8748 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:47.018206  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.035651  8747 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:47.036280 16073 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.036813  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.037890  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.038815  8823 raft_consensus.cc:2410] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 13:02:47.043743  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45403.6us
I20260709 13:02:47.045150  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.036755 16073 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:47.055919  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.058483  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:47.064889  8746 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
W20260709 13:02:47.066903  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45366.9us
I20260709 13:02:47.061866  8820 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:47.074815  8749 raft_consensus.cc:2368] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20.
W20260709 13:02:47.077416  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 131352us
I20260709 13:02:47.079726  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:47.079952  8744 raft_consensus.cc:2468] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:46.974244 15938 consensus_queue.cc:260] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.093552 15599 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:47.081132  8743 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:47.095306  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:47.094566  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 210203us
I20260709 13:02:47.092445  8745 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:47.098512  8747 raft_consensus.cc:1240] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:47.098728  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.095151  8741 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:47.102125  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.110235  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:47.111871  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 158111us
I20260709 13:02:47.116930  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.117928  8823 raft_consensus.cc:1240] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:47.114931  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.119809  8893 raft_consensus.cc:1240] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:47.120699 16025 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.121227 15626 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.121371 16025 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.122033 15626 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.124706 15626 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.124938 15599 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.125514  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:47.126205  8822 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.127223 15599 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.127849  8816 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:47.105654 15595 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.129200 15595 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.131630 15595 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.133589  8824 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:47.117722  8787 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:47.134387  9414 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.134924  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.135771  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:47.137362 15941 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:47.115465 16022 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.139763 16022 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.139777  8822 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:47.140599  8742 raft_consensus.cc:2393] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 25: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:47.141844 15626 raft_consensus.cc:2804] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:47.142835  8817 raft_consensus.cc:2410] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:02:47.143512  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.148775  8787 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:47.150996  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46375.7us
I20260709 13:02:47.150739  8740 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:47.157418  8824 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:47.157193 16090 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:47.160565 16083 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:02:47.129053 15826 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.162369 16091 consensus_queue.cc:1048] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:02:47.180418  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.123926 16025 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.152794 15938 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:47.186746  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 44794.2us
I20260709 13:02:47.176313 16090 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:47.209923  8863 leader_election.cc:336] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.800s (SENT)
I20260709 13:02:47.191129  8820 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:47.237908  8815 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2618) took 1516 ms. Trace:
W20260709 13:02:47.247973  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59696.6us
I20260709 13:02:47.248093 16092 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.190970 15626 raft_consensus.cc:697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.194767 16090 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.254356 15626 consensus_queue.cc:237] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.257920  8893 raft_consensus.cc:1240] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: []
W20260709 13:02:47.258872  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67079us
I20260709 13:02:47.247983  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.260239  8895 raft_consensus.cc:2393] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 21: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:47.261200  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.280843 16025 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.281464  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.281798 16025 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.282864 16022 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.283541 16022 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:47.284279  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.284667 16025 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.285589 16022 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.286455  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:47.190709 16091 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.273739 16090 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.281394 16089 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.301600  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 184961us
I20260709 13:02:47.242650  8815 rpcz_store.cc:276] 0709 13:02:45.693503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.102178 (+408675us) service_pool.cc:224] Handling call
0709 13:02:46.142586 (+ 40408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.143059 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.143091 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.634142 (+491051us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b2c0005b728. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:02:46.675482 (+ 41340us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.894977 (+219495us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:46.915678 (+ 20701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.941188 (+ 25510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.176150 (+234962us) raft_consensus.cc:1685] finished
0709 13:02:47.176175 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.209738 (+ 33563us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1514,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":29921,"spinlock_wait_cycles":67834240,"thread_start_us":219002,"threads_started":1,"wal-append.queue_time_us":485911}
W20260709 13:02:47.311331  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 63890.7us
I20260709 13:02:47.310073  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:47.315112  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.321393  8890 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:47.324039  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.325181 15949 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.325294  8742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa"
dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:56580
I20260709 13:02:47.326120  8742 raft_consensus.cc:606] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.325902 15949 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.326025  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.326555  8742 raft_consensus.cc:614] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:47.327493  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.327665  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.328313  8742 raft_consensus.cc:2376] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18. Re-sending same reply.
I20260709 13:02:47.328685  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.328905  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.329396  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.329919  8746 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:47.330071  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.330586  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.330969  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.331281  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.328336  8740 raft_consensus.cc:2393] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:47.328239  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 193314us
I20260709 13:02:47.331210  8742 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:47.334306  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.335543  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.339254  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.339637  8744 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:47.339963  8743 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.339988 16089 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.345557 16097 raft_consensus.cc:2804] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:47.346254 16097 raft_consensus.cc:697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.347311 16097 consensus_queue.cc:237] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.349968  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:47.350662 16053 consensus_meta.cc:287] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.693s	user 0.004s	sys 0.006s
I20260709 13:02:47.351961 16090 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:47.351933  8743 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
W20260709 13:02:47.357540  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 220346us
I20260709 13:02:47.362246  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:47.362529  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 52619.4us
I20260709 13:02:47.361187  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:47.368480  8863 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.898s (SENT)
I20260709 13:02:47.370375  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.371251 16094 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:47.371739 16094 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:47.372155 16094 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.368125 16090 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:47.378314 16032 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.378759 16032 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.378965 15826 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.379165 16032 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.382704 16089 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.384420 16096 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.385221 16096 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.387531 16096 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.387748 16053 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.388322 16053 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.389647  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.390550 16053 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:47.391947  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.394102  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.395267  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.396812  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.397502 15615 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:47.398309  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.399497 15949 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.332696 16073 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.408205 16073 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:47.410005 16082 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.280s	user 0.000s	sys 0.007s
I20260709 13:02:47.415026 15599 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:47.420559  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.422317 15970 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:47.422902 15970 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.423413 15970 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.434289  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.435563 16090 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:47.442723  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.444437  8890 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:47.444725  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:47.446375  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 87347.7us
W20260709 13:02:47.447638  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.453626  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.454303  8897 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:47.457450  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.459586  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.460394  8818 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:47.462381  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:47.463150  8815 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:47.459779  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.464038  8895 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:47.463152  8824 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 15.
I20260709 13:02:47.468735  8821 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:47.469550  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:47.470724  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 110143us
I20260709 13:02:47.470835  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:47.471432  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.450552 16025 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:47.471758  8817 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 16.
I20260709 13:02:47.471773 16096 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:47.472146  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.480222  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:47.493790  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:47.495211  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61068.8us
I20260709 13:02:47.486819 16073 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:47.479506 16032 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.480873 16094 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.514331  8894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815"
dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
mode: GRACEFUL
new_leader_uuid: "112105225cc140459688be5d31dcb071"
 from {username='slave'} at 127.0.0.1:53696
I20260709 13:02:47.515137  8894 raft_consensus.cc:606] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 112105225cc140459688be5d31dcb071
I20260709 13:02:47.515534  8894 raft_consensus.cc:614] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:47.501390 16073 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.516536  9414 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:47.521765  9449 raft_consensus.cc:1240] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:47.526439  9444 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:47.528478 16090 raft_consensus.cc:2804] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:47.529068 16090 raft_consensus.cc:697] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:47.530102 16090 consensus_queue.cc:237] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.535516  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.512786 16051 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.501550 16096 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:02:47.518991 16073 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:47.539568  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 179406us
I20260709 13:02:47.536839 15941 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.547046  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:47.523103 15884 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.538880 15615 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:47.542904 15975 consensus_queue.cc:1048] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:02:47.549823 16092 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.567726  8789 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.546010 16051 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.576489  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:47.576821 16032 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:47.577654  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50736.5us
I20260709 13:02:47.579547  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:47.548622 15823 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:02:47.589226 16094 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.552722 16099 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:02:47.571627 15884 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.593608  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:47.587301 16091 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.615715 15941 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.597142  8819 raft_consensus.cc:2468] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:02:47.598119  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.580014 15970 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.617452 16091 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.648501 15970 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.649750  9443 raft_consensus.cc:1240] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:47.649947 16101 log.cc:927] Time spent T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:02:47.581629  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.623147  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:47.651415  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:47.641171  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 133516us
I20260709 13:02:47.658030  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.659314  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.659621  8892 raft_consensus.cc:1240] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:47.661849  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.662017 15615 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:47.663043  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.663615  8889 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:47.668753  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.670749  8743 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.667795  9449 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:02:47.671555  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 90061.2us
I20260709 13:02:47.672567  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.673804  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:47.674386  9449 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:47.676065  8896 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:02:47.681341  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.685118  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.685730  8741 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.669859 16113 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:47.688374  8898 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:47.688568 16113 raft_consensus.cc:493] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.688956 16113 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:47.689853  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 91829.3us
I20260709 13:02:47.690183 15865 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.690697  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.692327 16096 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:47.649166 16091 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.705484  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:47.706756 16112 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.707669 16050 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:02:47.708398 16050 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:47.682895 16022 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:47.709594  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 183828us
I20260709 13:02:47.709867 16022 raft_consensus.cc:3055] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:47.710211 16022 raft_consensus.cc:740] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:47.711181 16022 consensus_queue.cc:260] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.712014  8715 leader_election.cc:400] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
I20260709 13:02:47.712338  8715 leader_election.cc:403] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:47.712595  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:47.667241 15884 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.713423  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.694880 15823 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:02:47.713189  8745 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.720875 16113 raft_consensus.cc:515] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.723582  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.725454 15987 consensus_queue.cc:1059] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:47.726749 15987 raft_consensus.cc:3055] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:47.727141 15987 raft_consensus.cc:740] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:47.727245 16091 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:47.728111 15987 consensus_queue.cc:260] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.671038 16104 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.670907 16025 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.695055 15949 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:02:47.739583  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 43559.6us
I20260709 13:02:47.741371  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:47.740420 16032 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:02:47.746484 16097 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.746198 16113 leader_election.cc:290] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.747284 16097 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.747841 16097 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.748762 15626 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.749195  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.749444 15626 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.749866  8896 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.751869 15626 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.751945 16097 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.753470 16025 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.755878 16025 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.763767  8896 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:47.762954  8749 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:47.765450  9412 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:47.766218  8743 raft_consensus.cc:2468] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:47.770097  8745 raft_consensus.cc:2468] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:47.776176  8896 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:02:47.780077  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 205490us
I20260709 13:02:47.740432 15987 raft_consensus.cc:3060] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.780795 16025 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.785930  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.793761  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.767186 16112 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:47.800884  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 87554.9us
I20260709 13:02:47.779628 16113 raft_consensus.cc:2804] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:47.821614  8741 raft_consensus.cc:2468] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
W20260709 13:02:47.801941  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 98993.6us
I20260709 13:02:47.803224  8788 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.805389 16104 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:02:47.781533 16073 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:47.829317  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.800856 15941 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:47.842521 16115 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:47.844269  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:47.844213 16115 consensus_queue.cc:1048] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:47.847185  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.847577 15626 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:47.851011 16123 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:47.851408 16053 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:47.851881 16053 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:47.852264 16053 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:47.853667  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 50563.7us
I20260709 13:02:47.853875 16073 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:47.854856  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.856549 15938 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.854779 15626 consensus_queue.cc:1059] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:47.842568 15941 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.859540 15941 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.863247 16051 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:47.857445 15982 consensus_queue.cc:1059] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:47.823993 16112 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.857651 15626 raft_consensus.cc:3055] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:47.866731 16112 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.867288 15626 raft_consensus.cc:740] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.856236  8788 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:47.867477  8863 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.868469 15823 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.868583 15626 consensus_queue.cc:260] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.828801 16113 raft_consensus.cc:697] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:47.869478 16097 raft_consensus.cc:2804] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:47.870080 16097 raft_consensus.cc:697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:47.870288  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.875988  8863 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.876327 15626 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:47.870672 16051 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:47.870210 16113 consensus_queue.cc:237] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.890025  9449 raft_consensus.cc:1240] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:02:47.876945 16053 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:47.879674 16022 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.880371 16096 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:47.880676  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:47.896297  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:47.896003  9443 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:47.871040 16097 consensus_queue.cc:237] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:47.895533  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.900944  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 177869us
I20260709 13:02:47.901227  8897 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:47.902556  8788 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.903219  9414 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:47.903555  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.905149  9414 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.907533  9414 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:47.911549  9414 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:47.913576  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:47.916220  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:47.896203 16096 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:47.921895  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 210135us
I20260709 13:02:47.917399  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.930470 15268 consensus_queue.cc:1059] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:47.931427 15268 raft_consensus.cc:3055] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:47.931773 15268 raft_consensus.cc:740] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:47.908722 16091 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:47.938012 16091 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.938416 16091 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:47.940888  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 60315.2us
I20260709 13:02:47.908798 15821 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.908645 15884 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.912964 15941 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.917245 16089 raft_consensus.cc:2804] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:47.908389 16092 raft_consensus.cc:2804] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:47.915750 15615 raft_consensus.cc:2804] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:47.932598 15268 consensus_queue.cc:260] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.958559 15884 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:47.944352 15823 consensus_queue.cc:1059] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 13:02:47.945540 16104 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:47.960922  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.961542 16089 raft_consensus.cc:697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:47.928830 16096 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.968808 16053 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:47.969437 16092 raft_consensus.cc:697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:47.898867 15865 consensus_queue.cc:1059] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:02:47.971025 16051 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:47.987031  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:47.989351  8741 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:47.956105 15941 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.978992 15615 raft_consensus.cc:697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:48.005762  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 118166us
I20260709 13:02:47.954870 15938 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:02:48.001402 15823 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.003093 15268 raft_consensus.cc:3060] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.996093 15865 consensus_queue.cc:1243] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 29c7934ee528454785432732d01ed0a0 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:02:48.003798 15941 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.015306 16095 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.021957 16100 raft_consensus.cc:3055] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:02:48.023691  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 69218.6us
I20260709 13:02:48.028120  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.009459 16032 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.040791  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 80043.2us
I20260709 13:02:48.044641  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.059929  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:48.060060 16135 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:48.066345  8815 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:48.030876 16097 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.039196 16100 raft_consensus.cc:740] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:48.079767  8817 raft_consensus.cc:1275] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
W20260709 13:02:48.079809  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 150685us
I20260709 13:02:48.077957  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.077883  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.060846 15987 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:48.080837  8817 pending_rounds.cc:85] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 15
I20260709 13:02:48.086880  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:48.088670  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 114993us
I20260709 13:02:48.095360  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.086697  8817 pending_rounds.cc:107] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:02:48.096149  8817 raft_consensus.cc:2889] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:48.102072 16132 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:02:48.103912 16129 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:02:48.104255 16073 raft_consensus.cc:3055] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:48.106810 15884 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.083704 16123 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.047648 16032 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.104878 16073 raft_consensus.cc:740] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:48.110633  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:48.090975  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:48.108815  8820 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:48.090930 16025 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18.
I20260709 13:02:48.123361 16100 consensus_queue.cc:260] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.123802  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.124771 16100 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:48.086144 16092 consensus_queue.cc:237] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.126480 16128 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.047936 16089 consensus_queue.cc:237] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.130482  8815 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:48.078629 16095 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.133558 16123 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.136792 16123 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.137810 16125 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:48.138621  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:48.085448 15615 consensus_queue.cc:237] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.127054 16128 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:48.162472  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76852.1us
I20260709 13:02:48.165606  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:48.169312  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58781.4us
I20260709 13:02:48.169569  8788 leader_election.cc:304] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.178807  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.180935 16134 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:48.181535 16134 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:48.181967 16134 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:48.187542  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:48.149271 15821 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.195235  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.196831  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:48.201884  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.202267  8788 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.202704  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:48.163663 16051 raft_consensus.cc:2764] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:02:48.209614  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.210790  8788 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.207193 15823 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:02:48.224125 16131 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:48.194972 15987 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:48.209702 16097 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.209017 16025 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.190657 16104 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:48.240680  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:48.244930  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:48.227828 16112 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:48.248028  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.232441 15987 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:48.252302  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 131121us
I20260709 13:02:48.253698 15941 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.281175 15941 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:48.281973 16141 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:48.257781  8745 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:48.283486  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:48.269531 15884 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:48.284368 15884 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.284806 15884 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:48.269802  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 68210.5us
I20260709 13:02:48.271382  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.274833 15823 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:02:48.275437  8863 connection.cc:256] RPC call timeout handler was delayed by -1.1401587520000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 13:02:48.276438 16051 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:48.277068 15821 consensus_queue.cc:1048] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:48.277946 15993 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:48.255074 16138 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:48.314687  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:48.321998  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 53711.4us
I20260709 13:02:48.328531  9443 raft_consensus.cc:1240] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:02:48.328722 16125 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:48.317265 15268 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:48.329777 15268 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.331998 15268 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.335093 16128 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.335206  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:48.336079 16136 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.336359  8864 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:48.336565 16136 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.336809 16100 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:48.336982 16136 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.337272 16100 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:48.337374 15975 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:48.338488 16100 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:48.338543 15975 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.339094 15975 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:48.341928  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:48.328547 16096 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.344870  8815 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:48.347931  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:48.348800 15975 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.349574 16096 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.350827  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.352864  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:48.353415  8817 raft_consensus.cc:2376] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24. Re-sending same reply.
I20260709 13:02:48.359175  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.360092  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.362716  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:48.363114  8819 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:02:48.302332 16025 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.239339 16073 consensus_queue.cc:260] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.373757 16099 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.379500 16100 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.384816 16099 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:48.385190 16073 raft_consensus.cc:3060] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.385363 16099 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.386327 16100 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:48.387130 16143 log.cc:927] Time spent T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:02:48.383895  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 112769us
I20260709 13:02:48.395653  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:48.385241  8898 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:48.406782  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:48.408103  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:48.358392 16095 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:48.411935  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 97375.4us
I20260709 13:02:48.413105  8864 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:48.414228  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:48.414180 16091 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.415046  8818 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:48.420699 16022 consensus_meta.cc:287] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
W20260709 13:02:48.425437 16154 log.cc:927] Time spent T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:48.425994 16134 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.428201 16134 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:48.429790  8863 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.602s (SENT)
I20260709 13:02:48.432636 16032 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:48.438098 15938 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:48.438305 15975 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.438572 15938 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:48.442598  8818 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
W20260709 13:02:48.445997  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.451129  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:48.451757  8741 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:48.377879 16025 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.454443  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:48.419349 16051 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.458335  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:48.458976  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65282.7us
I20260709 13:02:48.463269  9447 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:48.464534  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:48.444191 16115 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.468964 16115 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.471251 16115 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.436739 16136 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:48.477133  8863 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.798s (SENT)
W20260709 13:02:48.478307  8863 leader_election.cc:336] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.810s (SENT)
W20260709 13:02:48.479247  8863 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.826s (SENT)
I20260709 13:02:48.479615  8863 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:48.481015  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.439419 15938 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.486333 15938 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.487181 16136 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.488164  8897 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:48.489086 16157 consensus_queue.cc:1059] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:48.489500  9440 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:48.489856 16157 consensus_queue.cc:1243] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:02:48.490198  9448 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:48.490374 16157 raft_consensus.cc:3055] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:48.490736 16157 raft_consensus.cc:740] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:48.491887  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:48.493885  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:48.494626 15626 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:48.494601 16025 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.497906  9447 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:48.498848 15506 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.499280 15506 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.499637 15506 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.502849 16097 raft_consensus.cc:2749] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.503686 16131 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:48.412914 15987 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.505620  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:48.506246  9448 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:48.455425 15826 raft_consensus.cc:2804] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:48.491580 16157 consensus_queue.cc:260] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.508029  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:48.518839  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:48.521543  8715 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:48.521812  8741 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:48.482807  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 69777.5us
I20260709 13:02:48.541115  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:48.541854  8747 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.546036  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:48.482331 16032 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.507004 16138 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:48.555483  8864 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:48.557492 15268 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:48.558167 15268 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:48.559338 15268 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.482291 16099 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.564589  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:48.565399  8897 raft_consensus.cc:2393] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 21: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:48.567632 16099 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.570174  8747 raft_consensus.cc:2468] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:02:48.575256  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:48.546018 16022 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.553671 16025 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:48.626871  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 164069us
I20260709 13:02:48.636427  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
W20260709 13:02:48.627771  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 148576us
W20260709 13:02:48.637890  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.546877 16051 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.628860 16091 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.552974 15826 raft_consensus.cc:697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:48.640897  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 182689us
I20260709 13:02:48.550907 16115 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:02:48.656951  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:48.551734 16032 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.655154  9414 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.666571  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:48.668951  8817 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:48.667292  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:48.669607  8894 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:48.672375  8741 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:48.553385 15823 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:02:48.622622 16022 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.676877  8896 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:48.581184 16138 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:48.680498  8822 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:48.635527 16025 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:48.660108 16073 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:48.684516  8863 leader_election.cc:336] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.776s (SENT)
I20260709 13:02:48.579873 16151 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.588819 16136 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 13:02:48.654007 16157 raft_consensus.cc:3060] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.689610  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:48.678046 16053 consensus_queue.cc:1059] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:02:48.683617 16113 raft_consensus.cc:2804] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:48.689216 16025 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.686618 16138 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:48.694037 16158 log.cc:927] Time spent T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:02:48.697556  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61300.9us
I20260709 13:02:48.701182  8894 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 23: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:48.751734  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.753162 16125 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.753327  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:48.753940 16125 consensus_queue.cc:1059] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:48.754588  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:48.692202  8863 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:48.721132 15884 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.760846 16123 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:48.761279 16123 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:48.761680 16123 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:48.762195 15884 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:48.685025 16022 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.730130 16138 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.689682  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.738437  8741 raft_consensus.cc:1275] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:48.740759  8745 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:48.691589 16091 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:02:48.716329 16073 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:48.705696 15826 consensus_queue.cc:237] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.695487 16136 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:48.715705 16113 raft_consensus.cc:697] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:48.745975  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63408.1us
W20260709 13:02:48.765986  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.677488 16032 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:48.759549  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 68271.8us
I20260709 13:02:48.771142  8817 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
W20260709 13:02:48.810894  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 126422us
I20260709 13:02:48.793262 16073 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.795383 15975 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.795966 16136 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:48.704738 15506 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.812085 15823 raft_consensus.cc:3055] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:48.772374 16025 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.808246 16135 consensus_queue.cc:1048] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:02:48.873837  8745 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:48.877655  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 55248.2us
I20260709 13:02:48.810645 16092 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:02:48.829454 15823 raft_consensus.cc:740] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:48.899580 16136 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.903239 16025 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.903942 16025 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.904675 16025 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.899580 16051 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:48.905562 16051 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:48.884020 16125 raft_consensus.cc:3055] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:48.906494 16051 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.889595 16140 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:48.901117 16136 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:48.893333 15938 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.910252 16113 consensus_queue.cc:237] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.912160 16125 raft_consensus.cc:740] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:48.905083 16025 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:48.954790  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:48.965061 16167 log.cc:927] Time spent T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:02:48.967803 16135 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:48.968477 15970 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:48.968394 16135 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.906721 15823 consensus_queue.cc:260] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.969103 15970 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:48.969957 15823 raft_consensus.cc:3060] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.970988 16135 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.971410 15970 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:48.977522 16051 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.002264 16138 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.003561 15506 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:48.948500 15938 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.007297 15938 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:49.009495  8789 leader_election.cc:336] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.713s (SENT)
I20260709 13:02:49.009680 16025 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.010555  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:49.011528 16134 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:49.013504  8789 connection.cc:256] RPC call timeout handler was delayed by -1.0881114699999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 13:02:48.926653 16140 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.019905 16140 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:48.956363  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.025476  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 171309us
I20260709 13:02:49.032550  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.032774  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.032434 15626 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.033267  8819 raft_consensus.cc:3060] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.033594 15626 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.034276  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.035497  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.035681 15626 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.036679  8818 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:02:49.037014  8816 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:49.038398  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.040489  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:49.041188  9441 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.042135  8815 raft_consensus.cc:2410] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:49.043835  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.044685  8817 raft_consensus.cc:2393] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 16: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:49.046612  8821 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:49.046630  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.050187  8820 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
W20260709 13:02:49.052767  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.054158  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.054687 16022 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:49.055995  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.049718 16140 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.057555 16140 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.059248  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.055801 16152 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.067238  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 211734us
I20260709 13:02:49.077795  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.078562  8898 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.079828  8819 raft_consensus.cc:2468] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:49.082088  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.082688  8823 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.083670  8821 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:49.086898  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.089259  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.089876  8822 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:49.092916  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.066810 16178 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.961827 16125 consensus_queue.cc:260] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.099802  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.101511  8894 raft_consensus.cc:1275] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:02:49.108721  8889 raft_consensus.cc:1275] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:49.115738  8822 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
W20260709 13:02:49.116433  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 361945us
I20260709 13:02:49.117393  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.089783 16089 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.123539  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.126399  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.097345 16140 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:49.140290  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:49.140966  9448 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.079916 16152 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.142303  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.142875  8816 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 23: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:49.150753 16151 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:49.151984 16100 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:49.152412 16100 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:49.153668 16100 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.155237 16100 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:49.158241  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.161113  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ad465a724ab581b9529345f73bba" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.161878  8820 raft_consensus.cc:3060] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:49.166328  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49058us
I20260709 13:02:49.167703  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:49.166227  9443 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:02:49.104725 16091 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:49.189921  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.138075 16089 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.200048 16089 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:49.189365 16159 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:02:49.143808 16022 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.206101 16073 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.212358  8821 raft_consensus.cc:1275] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:49.213400  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.214597  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.215389  8821 raft_consensus.cc:2393] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 24: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:49.216133  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.220738  8818 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:49.221437 16179 consensus_queue.cc:1048] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 13:02:49.140694 16174 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:49.225121  8741 raft_consensus.cc:1240] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:49.217163  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:49.230046  8821 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 23: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:49.190719  8898 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:49.105000 16178 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.246057  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.247907  9448 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:49.246667  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.250946  8818 raft_consensus.cc:1275] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:02:49.251425  8818 pending_rounds.cc:85] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 13
I20260709 13:02:49.251700  8818 pending_rounds.cc:107] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:02:49.251981  8818 raft_consensus.cc:2889] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:49.272212  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.272687  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:49.231782 16174 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:49.290118  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122561us
I20260709 13:02:49.291118  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:49.245532 16073 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:49.300640  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 54060.8us
I20260709 13:02:49.248344 16022 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:49.254317 16125 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.307804  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:49.314332  8821 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:02:49.306981  8713 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:49.316507  8823 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:49.324659  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.310150  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.286779 16089 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:49.321899  8864 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:49.295424 16174 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.255450 16090 consensus_queue.cc:1048] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
W20260709 13:02:49.335330  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 44328.9us
I20260709 13:02:49.337775  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:49.339577  8821 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:02:49.302623 16181 raft_consensus.cc:2764] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:02:49.341410  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.339344 16179 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:49.344199 16173 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:49.344712 16173 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:49.345544 16173 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:49.358846  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.359503 16123 consensus_meta.cc:287] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.000s
I20260709 13:02:49.359974 16123 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.317845 16140 consensus_queue.cc:1059] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:02:49.362133 15970 raft_consensus.cc:3055] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:49.362620 15970 raft_consensus.cc:740] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:49.363654 15970 consensus_queue.cc:260] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.365116 15970 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.366994  9441 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:49.368273  8862 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:49.369104 16051 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:49.370244  8745 raft_consensus.cc:1240] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:49.372531  8820 raft_consensus.cc:2468] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:49.373782  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:49.375906  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65880.3us
W20260709 13:02:49.383412  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45766.4us
I20260709 13:02:49.340925 16176 raft_consensus.cc:2804] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:49.384533  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:49.391741  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.401970  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:49.375015 16152 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.380878 16073 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.382308 16051 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:49.369642 15506 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:49.414657  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:49.405027 16032 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.591s	user 0.005s	sys 0.009s
I20260709 13:02:49.416608 16181 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:49.417227 16094 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:49.417656 16135 consensus_queue.cc:1048] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:49.425861  8749 raft_consensus.cc:1240] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:49.426491  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.442839  8740 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:49.428228  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.444010  8742 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:49.430771 16051 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.431625 16022 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:49.406128 15993 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.440387  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 118534us
I20260709 13:02:49.405067 16176 raft_consensus.cc:697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:49.511056  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:49.511441  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 178181us
W20260709 13:02:49.511699  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 109794us
I20260709 13:02:49.514115  9443 raft_consensus.cc:3060] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:49.515380  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.516085  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.516556  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.515988  9440 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
W20260709 13:02:49.517554  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 66471.1us
I20260709 13:02:49.442884 16032 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.539089  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.539714  8891 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:49.520828  8892 raft_consensus.cc:2410] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:02:49.457207 15993 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.515944 16178 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.511111 16123 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.529695 16173 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:49.534446  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:49.541463  8820 raft_consensus.cc:1240] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:49.539855  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:49.538659 16171 log.cc:927] Time spent T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:49.563189  9447 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:49.546162 16104 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.546381 16022 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.509323 16176 consensus_queue.cc:237] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:49.574584  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 160022us
I20260709 13:02:49.537914 16025 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:02:49.517712 16181 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:02:49.562610 16178 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:49.558166 16173 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:49.563701 16090 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.571072 16136 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.001s
I20260709 13:02:49.567910 16197 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.618979 16197 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.621126 16197 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.621451 15993 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.572175 16135 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:49.622014 16135 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.622414 16135 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:49.615216 16178 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.625847 16178 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.574321 16022 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.629022 16022 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.589640  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.574973 16113 consensus_queue.cc:1059] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:02:49.630352  9444 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:02:49.617955 16090 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.632445 16090 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.630304  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.634122  9415 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.634931 16091 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.635857 16091 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:02:49.636906 16180 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.309s	user 0.004s	sys 0.000s
W20260709 13:02:49.637900 16196 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:02:49.638227 16032 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:49.639856  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.640503  9449 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:49.574807 16104 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.641484  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.642354  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:49.641531  9415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.643244 16104 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.643520 16135 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.649042 16135 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:49.616513 16173 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.630543 16136 raft_consensus.cc:3055] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:49.665984 16136 raft_consensus.cc:740] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:49.672951  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.689675  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 123178us
W20260709 13:02:49.691519  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.696239  8712 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:49.697271  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 128053us
I20260709 13:02:49.700583  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:49.702109  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.702760  8891 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
W20260709 13:02:49.703357  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 187356us
I20260709 13:02:49.704501  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.706710 15970 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:49.707628  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.674712 16125 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:49.705725 15938 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:02:49.734879  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 475036us
I20260709 13:02:49.735422 15938 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:49.718362  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.737668  9447 raft_consensus.cc:2393] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 24: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:49.738595 15938 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:49.739926 15938 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.743690 15938 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.745479 16051 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:49.716536 16073 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:49.716671 16025 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:49.703833 16181 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:49.716998 16094 consensus_queue.cc:1059] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:02:49.723173 16209 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.752686 16025 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:49.754015 16073 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.723728 16174 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.756851 16094 raft_consensus.cc:3055] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:49.728859 16190 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 13:02:49.667279 16136 consensus_queue.cc:260] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.733798 16125 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.759821 16125 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.757402 16175 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:02:49.763698 16152 consensus_queue.cc:1059] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:49.764874 16175 raft_consensus.cc:3055] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:49.757263 16094 raft_consensus.cc:740] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:49.767225 16173 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.768703 16094 consensus_queue.cc:260] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.769464 16136 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.769941 16173 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.770442 16094 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:49.735651  8789 leader_election.cc:336] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.805s (SENT)
I20260709 13:02:49.773000 16125 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.775667 16125 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.779206 16178 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.781092 16140 consensus_queue.cc:1048] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:49.791069 16210 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:49.791855 16210 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.793547  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.751273  9443 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:02:49.714541  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:49.798460  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84018.3us
I20260709 13:02:49.798825  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:49.751703  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.755330 16209 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.724445  8824 raft_consensus.cc:1275] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:02:49.733065 16100 consensus_meta.cc:287] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.002s
I20260709 13:02:49.759884 16025 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.803570 16178 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.807170  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.808063  8743 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:49.760624 16181 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.809634 16181 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.803252 16125 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:49.810648 16125 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:49.811074 16125 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.812121 16181 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.758049 16174 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.759624 16073 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.814173  8748 raft_consensus.cc:1275] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:49.814589  8748 pending_rounds.cc:85] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:49.814802  8748 pending_rounds.cc:107] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:49.815019  8748 raft_consensus.cc:2889] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.767235 16207 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.816468  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.816409 16207 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.819168 16207 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.767452 16175 raft_consensus.cc:740] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:49.799484  9440 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.820704 16175 consensus_queue.cc:260] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.822395 16175 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.823400 16210 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:49.826045  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.829605  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.830555  8749 raft_consensus.cc:2410] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:02:49.834575  8744 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:49.835201  9440 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:49.836404  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:49.837024  9440 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.802903  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.837989  8745 raft_consensus.cc:3060] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.805074  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.815886 16209 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.838909  8742 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:49.837054  8713 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:49.838560  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.842103  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.843073  8742 raft_consensus.cc:2368] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:49.842124  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:49.852267  8741 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:49.854583  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:49.859596  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 69115.5us
W20260709 13:02:49.860817  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 95906.7us
I20260709 13:02:49.863430  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.869719  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.872607  8898 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:49.870267  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.873829  8894 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
W20260709 13:02:49.872821  8789 leader_election.cc:336] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.924s (SENT)
I20260709 13:02:49.874495  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:49.875937  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.875845  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.876644  8742 raft_consensus.cc:2368] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:49.877264  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.878613  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.878551  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.879704  8741 raft_consensus.cc:2368] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 22. Current term is 23.
I20260709 13:02:49.884013  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:49.843286 16207 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:49.886165 16207 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:49.887265 16207 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.878355 16138 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.870868 16100 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
W20260709 13:02:49.891155  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.892617 16208 consensus_queue.cc:1048] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:49.862557 16125 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.893937  8897 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:49.894783 16138 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:49.895184 16138 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.895696 16138 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:49.899894 16192 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.390s	user 0.003s	sys 0.000s
I20260709 13:02:49.861799 16113 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:49.903043 16025 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:49.895217 16125 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:49.907207  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.915633  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.909233  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.910362  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.916723  8896 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.916802  8747 raft_consensus.cc:2468] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:49.911422  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.917673  8744 raft_consensus.cc:2468] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:49.908737  9440 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:49.912096  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.918627  8742 raft_consensus.cc:2468] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:49.912827  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.913316  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.913666  8748 raft_consensus.cc:1275] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:02:49.919514  8897 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.912168  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.918974  8713 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:49.930106  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:49.930848  8890 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 25.
I20260709 13:02:49.931124  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:49.932144  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 131915us
I20260709 13:02:49.905267 16025 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:49.940385  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79779.6us
I20260709 13:02:49.940510  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:49.941090  8789 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.108s (SENT)
I20260709 13:02:49.941476  8741 raft_consensus.cc:2393] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 16: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:49.938165  9412 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:49.947866  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.936493  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.561s (SENT)
I20260709 13:02:49.926996 16096 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.958025 16136 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:02:49.961799  8745 raft_consensus.cc:2468] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:49.964769  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:49.970786  8743 raft_consensus.cc:1275] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:49.973098  8742 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:49.974083  8897 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:49.976125  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.976054  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.977501  8741 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:49.974646  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:49.949199 16125 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:49.975565  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.985988  8749 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.988752  8896 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:49.959012 16073 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:49.992844  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:49.993868  8748 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:49.972921 16136 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.991786 16207 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.005981  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64935.8us
I20260709 13:02:50.010828  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.012871  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.013615  8818 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:50.016106  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.016875  8817 raft_consensus.cc:2368] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21.
I20260709 13:02:50.020179  8741 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:50.022964  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:50.023159  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 86728.3us
I20260709 13:02:49.994674 16125 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:50.031262 16125 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.005697 16136 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:50.023600 16207 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:49.975653 16025 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:02:50.038898  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:50.039177  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.043326  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:50.043948  8863 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 1.618s (SENT)
I20260709 13:02:50.040699  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:49.972710 16096 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.040055  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.044780  8747 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:50.045621  8819 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:50.046756  8745 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:50.048007  8745 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:50.050391  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.050979  8747 raft_consensus.cc:2376] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18. Re-sending same reply.
I20260709 13:02:50.053067  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.044111  8741 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:50.057966  8818 raft_consensus.cc:2410] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:02:50.058442  8744 raft_consensus.cc:1275] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:50.055419  8746 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:50.061643  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.066470  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.068189  8741 raft_consensus.cc:2376] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21. Re-sending same reply.
W20260709 13:02:50.068817  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.072003 16206 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.086274  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.087096  8740 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:50.063131 15826 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.084812  8749 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:50.103132  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.107734  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.108982  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.118711  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.120144  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.120921  8749 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:50.101085 16206 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.156316  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.114508 16073 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.179817  8818 raft_consensus.cc:2393] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:50.123870 16174 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.186954 16216 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:02:50.187395  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.187745  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.188272  8822 raft_consensus.cc:2393] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:50.188864  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.189118  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.190035  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.100592 15826 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.200896  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.197305  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.202126  8747 raft_consensus.cc:3060] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:50.204252  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.206225  8743 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:50.206527  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.196101  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:50.210517 16032 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:50.211038  8818 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 13:02:50.206211  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:50.223987  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.234763  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.235430  8741 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 25.
W20260709 13:02:50.238008 16224 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:50.195415 16174 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.238116  8822 pending_rounds.cc:85] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:02:50.239461  8822 pending_rounds.cc:107] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:02:50.239773  8822 raft_consensus.cc:2889] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.233376  8747 raft_consensus.cc:2468] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:02:50.240155  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:50.215821 16223 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:50.181731 16206 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.183671 16073 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.304947 16207 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.312377 16206 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:50.316046 16138 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.243676 16208 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.323014  8746 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.342545  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:50.304950 16222 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:02:50.349050 16096 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:50.357764 16175 consensus_meta.cc:287] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.011s
I20260709 13:02:50.347278 16208 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.285940  8742 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:50.368299  8891 raft_consensus.cc:2410] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:02:50.371403 16229 log.cc:927] Time spent T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:02:50.369326 16174 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:50.385866 16073 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.388749  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:50.374961 15826 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.390895  9443 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:50.416194 16208 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:50.424816 16235 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:50.426000 16138 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:50.429891  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.430441  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.431135  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:50.442711 16136 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.445554 16136 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:50.470889  8746 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
W20260709 13:02:50.531188 16227 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.340s	user 0.000s	sys 0.010s
W20260709 13:02:50.555049 16226 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.356s	user 0.007s	sys 0.032s
W20260709 13:02:50.562162  8789 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.165s (SENT)
W20260709 13:02:50.563084 16220 log.cc:927] Time spent T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.486s	user 0.004s	sys 0.012s
W20260709 13:02:50.563412  8789 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.170s (SENT)
W20260709 13:02:50.564970  8789 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.186s (SENT)
I20260709 13:02:50.565058  9414 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:50.568740 16174 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.570724  8740 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:50.571352  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.573989  8748 raft_consensus.cc:1275] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:02:50.603689  8748 pending_rounds.cc:85] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 10
I20260709 13:02:50.603927  8748 pending_rounds.cc:107] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:02:50.604190  8748 raft_consensus.cc:2889] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.574769  8745 raft_consensus.cc:1240] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:50.575671  8747 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:50.571374  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.578552  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.606472  8816 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
W20260709 13:02:50.583441  8864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.607115  8745 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:02:50.607347  8744 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 18: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:50.584013  8743 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:02:50.585525  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.608439  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.608512  8819 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:50.587338  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:50.609272  8740 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:50.609630  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:50.609944  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.591806  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.610976  8644 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:50.612864  8747 raft_consensus.cc:1217] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 19.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 13:02:50.614007  8745 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:50.602474 15826 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.614784 15826 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:50.594053 16215 log.cc:927] Time spent T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.620s	user 0.008s	sys 0.028s
I20260709 13:02:50.594169 16208 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:50.617631 16208 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:50.617944  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53015.5us
I20260709 13:02:50.605142 16113 raft_consensus.cc:2804] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:50.618752 16209 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.607370 16174 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.619736 16208 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:50.618821 16113 raft_consensus.cc:697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:50.620342 16208 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.620930 16113 consensus_queue.cc:237] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.620837 16208 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.623914 16208 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:50.597596  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.607828  8742 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:50.617854 15826 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.627524  9414 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:50.625586  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.625766  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.625926  8747 raft_consensus.cc:1240] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:02:50.642444  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.643029  8747 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:50.578893  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:50.618708  8789 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.190s (SENT)
I20260709 13:02:50.627493  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.629511  8742 raft_consensus.cc:1240] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:50.630978  8748 raft_consensus.cc:1240] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:50.649086  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.649652  8743 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:50.646219  8644 catalog_manager.cc:5697] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 22 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.652570  8746 raft_consensus.cc:2376] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22. Re-sending same reply.
I20260709 13:02:50.621068 16174 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:50.663708  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.624423 16208 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.670425  8747 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 26.
I20260709 13:02:50.669745 16096 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:50.677186  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58511.9us
W20260709 13:02:50.677992  8789 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.191s (SENT)
I20260709 13:02:50.678627  8644 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 19 to 22, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 22 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.679414  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.680462 16096 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.681520 16125 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.682142 16125 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.683532  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.684613  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.685664  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.685873  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.686671  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.686857  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.656733 16140 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:50.687830  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.688194  8748 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:50.688082 16140 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:50.680378  8644 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 22, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:50.689466  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.698783  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.699503  8823 raft_consensus.cc:2410] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:50.699720  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.700305  8742 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:50.693419  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.689239  8740 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:02:50.693713  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.701934  9415 leader_election.cc:400] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21.
I20260709 13:02:50.702255  9415 leader_election.cc:403] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:50.702383  8743 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 23: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:50.703135  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.697659  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.695842  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:50.704711  8748 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:50.706234  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:50.707142  8790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.707090  8789 leader_election.cc:336] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.519s (SENT)
I20260709 13:02:50.707721  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:50.707813  8789 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.708642  9444 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 23.
I20260709 13:02:50.709961  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:50.711968  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.712863  8819 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:50.715561  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:50.716333  8896 raft_consensus.cc:3060] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:50.718372  8743 raft_consensus.cc:3055] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:50.718869  8743 raft_consensus.cc:740] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:50.710517  8748 raft_consensus.cc:2376] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20. Re-sending same reply.
I20260709 13:02:50.736039  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.703835  8740 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:50.719823  8743 consensus_queue.cc:260] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.741221  8743 raft_consensus.cc:3060] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.742137  8644 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 21, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.709887 16125 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.714247 15987 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.747046  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48537.9us
I20260709 13:02:50.714190 16096 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.747280  8644 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:50.749423  8789 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Timed out: RequestConsensusVote RPC to 127.7.117.129:39991 timed out after 2.689s (SENT)
I20260709 13:02:50.738046 16135 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 20. Current term is 21.
I20260709 13:02:50.750828  8863 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:50.748775  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 87013.9us
I20260709 13:02:50.690980 16208 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.752382  9414 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:50.754258  9415 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.755692  8644 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 18, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:50.766258  8896 raft_consensus.cc:2468] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 26.
I20260709 13:02:50.767151  8743 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:50.767524  8743 pending_rounds.cc:85] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 9
I20260709 13:02:50.767751  8743 pending_rounds.cc:107] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:50.767999  8743 raft_consensus.cc:2889] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.769091  8889 raft_consensus.cc:1275] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:02:50.769662  8889 pending_rounds.cc:85] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Aborting all ops after (but not including) 13
I20260709 13:02:50.769950  8889 pending_rounds.cc:107] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:02:50.770269  8889 raft_consensus.cc:2889] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.769433  8644 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 8 to 22. New cstate: current_term: 22 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.771401  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.690280 16140 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.748569 16089 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:50.776477 16207 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:50.776912  8645 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 287) took 31616 ms (client timeout 15000 ms). Trace:
I20260709 13:02:50.751065 15826 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:50.738125 16176 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:02:50.778999  9444 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:50.760664 16179 raft_consensus.cc:2804] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:02:50.779649  9443 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:50.779670 16176 consensus_queue.cc:1059] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:50.780958  8644 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 19, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 19 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:50.775390  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:50.784240  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.786736  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.787128  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:50.782385  8644 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 20 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:50.787948  8819 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 23.
I20260709 13:02:50.760444 16202 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.788733  8644 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 18, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 18 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.794953  8644 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 16, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 16 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:50.782992 16242 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.751745 15987 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.785753 16138 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:50.805809 15987 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:50.805753 16138 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:50.806531 15987 raft_consensus.cc:3055] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:50.806895 15987 raft_consensus.cc:740] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:50.808210 16138 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:50.807885 15987 consensus_queue.cc:260] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:50.810578 16206 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.813324  9443 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:50.820109  8789 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.789264 16025 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:50.791498 16179 raft_consensus.cc:697] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 26 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:50.839493  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:50.867226  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:50.867839  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 47865us
I20260709 13:02:50.868884  8789 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.799991  8644 catalog_manager.cc:5697] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 6 to 21. New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:50.871268  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 84849.4us
I20260709 13:02:50.871237  8644 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 19, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 19 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.872123  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.873359  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.876806  8644 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 16, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 16 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:50.878839  8644 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.880897  8644 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 16, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 16 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:50.888931  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 119220us
I20260709 13:02:50.890484  8644 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 15, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 15 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.786870 16125 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:02:50.789456 16207 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:02:50.905802  8644 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 21, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:50.875254 15941 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:02:50.907562  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 142767us
W20260709 13:02:50.913920  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45163.5us
I20260709 13:02:50.912868  8644 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 16, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 16 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.839183 16209 consensus_queue.cc:1048] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:02:50.923297  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50147us
I20260709 13:02:50.866732 16096 consensus_queue.cc:1059] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:02:50.924583  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:50.839679 16206 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 13:02:50.874090 16138 consensus_queue.cc:1048] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:50.914526 16174 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:50.938810 15941 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.789600  8645 rpcz_store.cc:269] 0709 13:02:19.160243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.160415 (+   172us) service_pool.cc:224] Handling call
0709 13:02:19.271239 (+110824us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b5400002210. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33ab640 00007f90fc282b76 00007f90fc38cc3e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:02:50.776684 (+31505445us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:46.447123 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:46.447698 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:46.447737 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:02:46.447755 (+    18us) tablet.cc:662] Decoding operations
0709 13:02:46.460567 (+ 12812us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:46.464088 (+  3521us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:46.465612 (+  1524us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:02:46.470889 (+  5277us) tablet.cc:701] Row locks acquired
0709 13:02:46.470914 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:02:46.471056 (+   142us) write_op.cc:270] Start()
0709 13:02:46.471141 (+    85us) write_op.cc:276] Timestamp: P: 1783602166471017 usec, L: 0
0709 13:02:46.471155 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 13:02:46.629417 (+158262us) log.cc:844] Serialized 8714 byte log entry
0709 13:02:46.831462 (+202045us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cae28 00007f90f9f05581 00007f90f9f04d58 00007f90f9f099f1 00007f90f9f099a9 00007f90f9f09939 00007f90f9f09901 00007f90f9f08b2d 000000000038f354 000000000038f199
0709 13:02:47.119982 (+288520us) op_driver.cc:464] REPLICATION: finished
0709 13:02:47.377134 (+257152us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f9f05ef0
0709 13:02:47.743508 (+366374us) write_op.cc:301] APPLY: starting
0709 13:02:47.743622 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:48.050343 (+306721us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:48.072769 (+ 22426us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:48.711235 (+638466us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:48.711680 (+   445us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:02:48.711754 (+    74us) write_op.cc:312] APPLY: finished
0709 13:02:48.712285 (+   531us) log.cc:844] Serialized 196 byte log entry
0709 13:02:48.824784 (+112499us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.944258 (+119474us) write_op.cc:454] Released schema lock
0709 13:02:48.944355 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:48.944843 (+   488us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.944883 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11050,"reported_tablets":48,"spinlock_wait_cycles":5938944,"child_traces":[["op",{"apply.queue_time_us":611195,"apply.run_cpu_time_us":71745,"apply.run_wall_time_us":1201333,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96043,"num_ops":22,"prepare.queue_time_us":251170,"prepare.run_cpu_time_us":35414,"prepare.run_wall_time_us":384560,"raft.queue_time_us":253972,"raft.run_cpu_time_us":24722,"raft.run_wall_time_us":456824,"replication_time_us":645254,"spinlock_wait_cycles":256093696,"thread_start_us":834212,"threads_started":5,"wal-append.queue_time_us":381591,"wal-append.run_cpu_time_us":10150,"wal-append.run_wall_time_us":2194708}]]}
I20260709 13:02:50.918905  8644 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 23, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 23 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.943110 16256 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.940099 16175 consensus_queue.cc:1048] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 13:02:50.975880  8714 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.984702  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.955935 16125 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.961104 16179 consensus_queue.cc:237] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.014503  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.959414 16251 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:02:50.961565 16208 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:50.960817 16096 raft_consensus.cc:3055] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:50.958923 16138 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.939349 15941 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.026201  8789 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:50.943943 16089 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.023512 15987 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:51.035789  8749 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:02:51.044235  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.045984  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.036780 16096 raft_consensus.cc:740] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:51.067067  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.033473  8644 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 15, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 15 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.086239  8862 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:51.086496  8644 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 13 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:51.093122  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.088332  8644 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 17, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 17 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:51.093803  8889 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:51.094982  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.049724 16257 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:51.047272 16259 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:51.098498  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 174033us
I20260709 13:02:51.096673  8889 raft_consensus.cc:1240] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.066282 16136 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.056442 16242 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.099283  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:51.097904  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 207162us
W20260709 13:02:51.100212  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 297714us
I20260709 13:02:51.102681  8894 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:51.100999  8644 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 19, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 19 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:51.085765 15993 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:51.086325 16261 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:51.110219  8644 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 17, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 17 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:51.050501 16089 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.113749  8644 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 2 to 10, term changed from 13 to 21, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132), 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:51.119313  8644 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:51.122942  8894 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:51.121765  8644 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 18, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 18 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.124163  8895 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:51.125922  8644 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 16, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 16 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:51.114769 16269 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.115918 16261 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.117376 16259 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.107787 16136 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:51.143322  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 76349.4us
I20260709 13:02:51.137480  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.161327  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.115694 15993 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.140357 16258 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:51.144522 16261 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:51.135197 16096 consensus_queue.cc:260] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.167536 16266 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:02:51.144657 16269 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.191887 16280 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:51.192824 16269 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:51.193430 16269 consensus_queue.cc:1048] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:51.194145 16280 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:51.194337  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:51.194731 16280 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.195283 16280 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.136690 16264 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:51.198776 16264 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.199244 16264 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.200119  8896 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.200626 16266 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:51.201133 16266 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.201606 16266 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.155598 16259 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.189497 16136 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:51.198473  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.168643 16258 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.203653 16258 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:51.172478  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.178117 16096 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.204380  8895 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 23.
I20260709 13:02:51.181806 16089 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.182740 15993 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:51.206384 15993 consensus_queue.cc:1059] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:51.207401 15993 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.208031 15993 raft_consensus.cc:3055] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:51.208381 15993 raft_consensus.cc:740] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:51.173566  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:51.210189  8819 raft_consensus.cc:606] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.210599  8819 raft_consensus.cc:614] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.187361 16273 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.213196 16273 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.214355  8819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:51.215119  8819 raft_consensus.cc:606] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.215524  8819 raft_consensus.cc:614] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.192740 16175 consensus_queue.cc:1059] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:02:51.218183 16175 raft_consensus.cc:3055] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:51.218523 16175 raft_consensus.cc:740] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:51.219256 16175 consensus_queue.cc:260] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.222955  8863 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.228835  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.233490  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:51.234283  8749 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:51.240866  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.245362  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:51.237751  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:51.248590  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:51.249080  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 172886us
I20260709 13:02:51.249481  8819 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:51.255342  8819 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:51.255255  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:51.263727  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 164551us
I20260709 13:02:51.264660  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.265213  8816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:51.266047  8816 raft_consensus.cc:606] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.266541  8816 raft_consensus.cc:614] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.237706 16244 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.271435 16256 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:51.274765  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.231329 16257 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:51.275748  8712 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:51.278627  9440 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:51.251896 16175 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.279438  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.281385  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.281983  8823 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:02:51.257027  8863 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.283416  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:51.283512  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:51.280684  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:51.284355  8896 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:51.251613 15941 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:51.262240 16176 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.302445  8715 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.254976 16253 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.303982  8816 raft_consensus.cc:1275] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:02:51.301164  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 139956us
I20260709 13:02:51.252856 16140 consensus_queue.cc:1048] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 13:02:51.309569  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52648.9us
I20260709 13:02:51.313244  8896 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:51.314649  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.209206 15993 consensus_queue.cc:260] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.310477  8822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "b60591217ee24378bb66a86d6d0dda52"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:51.315717  8822 raft_consensus.cc:606] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Received request to transfer leadership to TS b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.311481  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.311218  8789 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:51.317619  8715 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.280391 16263 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:51.293025 16260 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:51.320683  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.302954 16073 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:51.303107 16250 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.352326 16094 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.313850 16206 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.318166 16273 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:51.375607  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 64507.1us
I20260709 13:02:51.384652  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:51.379900  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62368.9us
I20260709 13:02:51.321667 16260 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.374042 16250 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.387267  8816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
new_leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
 from {username='slave'} at 127.0.0.1:44786
I20260709 13:02:51.387895  8816 raft_consensus.cc:606] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Received request to transfer leadership to TS 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.388235  8816 raft_consensus.cc:614] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:51.388624 16282 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.389313 16282 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.389557  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:51.389778 16282 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.391307  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:51.391045  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:51.429373  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44809.3us
I20260709 13:02:51.429656 16089 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.430600  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.320254 16259 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.315284 16256 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:02:51.388972 16250 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.324723 16278 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.440574  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:51.448462  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 62373.7us
I20260709 13:02:51.525918  8715 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:51.436116  8864 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:51.527086  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 91076us
I20260709 13:02:51.528678  8864 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:51.384761 16209 raft_consensus.cc:2764] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:02:51.448225 16282 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:51.377743 16263 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.376500 15993 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.443406 16257 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:51.463166 16281 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:02:51.448668 16273 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.438815 16288 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 13:02:51.318949 16253 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.450184 16073 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.353358  8787 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.384344 16094 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.479765 16138 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:51.480988 16260 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:51.513120  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 211566us
I20260709 13:02:51.514957  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.485369 16261 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:51.531545  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 216968us
I20260709 13:02:51.532033  9444 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:51.536805  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:51.543503  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 190299us
I20260709 13:02:51.556838  8787 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:51.509579 16264 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.533915 16206 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:51.563973  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.563843  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.564162  8819 raft_consensus.cc:1240] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:51.562237  8749 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
W20260709 13:02:51.560859  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46010.6us
I20260709 13:02:51.519438 16089 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.565948  8789 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.567373  8863 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.553995 15941 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:51.552984 16282 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:51.561563 16207 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:51.587222 16138 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:51.600059  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 63388.7us
I20260709 13:02:51.600991  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:51.605744  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.563375 16209 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:51.614668  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48815.2us
I20260709 13:02:51.615525 16274 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.608989 16280 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.600836 16282 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.616772  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.599795 15826 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:51.629312 15826 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.565994 16263 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:02:51.632104  8822 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:51.608749 16206 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.634824 16206 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:51.615665 16265 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.609679 16138 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.636149 16265 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.636646 16265 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.637612 16138 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:51.627138 16089 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.589974 15987 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:02:51.643327 16207 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:51.643831 16207 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.644194 16207 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:51.645593  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.607313 16303 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:02:51.568908 16051 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:02:51.656266  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 50639.3us
I20260709 13:02:51.678704  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.680687 16208 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:51.681144 16208 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:51.682046 16208 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.683367 16208 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.621433 16283 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:51.670501  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53806.7us
I20260709 13:02:51.646261 16282 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.607259 16299 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.622421 16300 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:51.648245  8744 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:51.622910 16096 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:51.616163 16274 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.624213 16278 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:51.673817 16289 log.cc:927] Time spent T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:02:51.735991  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57394.2us
I20260709 13:02:51.740255  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:51.646055 16209 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:51.748943  8746 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:02:51.748342  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.673669 16025 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.645668 16294 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:51.648020 15826 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:51.755652  8789 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:51.687742 16138 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:51.755977  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 188489us
I20260709 13:02:51.763572  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.607470 16258 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.751351 16174 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:51.780694  8863 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:51.673429 16202 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 13:02:51.748304 16283 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:51.789343  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.789887  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.790602  8821 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:02:51.792429  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:51.795365  8821 pending_rounds.cc:85] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 13
I20260709 13:02:51.795933  8821 pending_rounds.cc:107] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:02:51.796242  8821 raft_consensus.cc:2889] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.759073 16256 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:51.793712  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.795337  8823 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:02:51.794956  8815 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:51.792474  8818 raft_consensus.cc:1275] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:02:51.798720  8818 pending_rounds.cc:85] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 15
I20260709 13:02:51.798976  8818 pending_rounds.cc:107] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:02:51.799036  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.799286  8818 raft_consensus.cc:2889] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.799736  8824 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:02:51.801344  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.801910  8824 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:51.804451  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:51.782467 16138 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:51.754030 16257 consensus_queue.cc:1048] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:02:51.720286 16266 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.788192 16273 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.808213  8863 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.808887  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:51.813273  8890 raft_consensus.cc:1275] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:02:51.803052 16261 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.801803 16285 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:51.811872 16257 consensus_queue.cc:1243] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer b60591217ee24378bb66a86d6d0dda52 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:51.817171 16094 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:51.816686 16273 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.920744 16273 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:51.918066 16314 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:02:51.918954 16305 log.cc:927] Time spent T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:02:51.900707 16051 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:02:51.835999 16209 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:51.925510 16253 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.927875 16285 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
W20260709 13:02:51.929307  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 110094us
I20260709 13:02:51.766752 16300 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.811460 16311 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.809656 16138 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.884011 16280 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:51.804683 16256 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:51.910569 16264 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:51.984620  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 176491us
W20260709 13:02:51.987519  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55813.2us
I20260709 13:02:51.989750  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:51.904464 16136 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.095s
I20260709 13:02:51.987545 16263 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.988227 15826 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:52.025825 16318 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:52.026569 16318 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.027019 16318 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:52.047533  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57441us
I20260709 13:02:51.925030 16283 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:52.050225 16304 log.cc:927] Time spent T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260709 13:02:52.055734 16279 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:52.056702 16310 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:02:52.079681  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.058044 16266 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.064950  8864 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:52.066871 16265 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.070027 16282 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.073150 16274 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.074934 16300 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.076277  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.093763  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.094496  8897 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:52.076548  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.077169  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:51.974841 16311 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.077689 16303 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.100122  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:52.052986  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63318.9us
I20260709 13:02:52.088697  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.102041  8749 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:52.096860  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:52.098351  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 287803us
I20260709 13:02:52.099980  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.077920  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.109740  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.114903  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.115720  8823 raft_consensus.cc:3060] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.117746  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.109719  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.120822  8822 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.086426 16136 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.089630 16089 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.089600 16202 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.138974  8744 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.077859 16285 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.142836  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.146984  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.143301  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.143580  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.148177  8741 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.150828  8748 raft_consensus.cc:1240] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:52.034612 16263 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.114446 16312 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:52.155503  8821 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:52.104184 16258 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.160502  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.147655  8749 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:52.162906  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.166865  8742 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:52.165827  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.167514  8748 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:52.143095 16179 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:52.175876  8822 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:52.149200 16136 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.150642 16202 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.196141 16256 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.159032 16312 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.196472 16265 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.109081 16303 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:52.197219 16260 consensus_meta.cc:287] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.634s	user 0.000s	sys 0.010s
I20260709 13:02:52.199494 16273 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.150484 16089 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.200824  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.251520  8745 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.251850  9444 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 22.
I20260709 13:02:52.154765 16250 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:02:52.193650 16302 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.203838 16311 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.189878 16291 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.120038 16266 consensus_queue.cc:1059] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:02:52.205554  8891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 3071) took 1113 ms. Trace:
I20260709 13:02:52.137712 16274 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:52.213872  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 61084.6us
W20260709 13:02:52.213932  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 53122.2us
I20260709 13:02:52.214498  8741 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:52.267000  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.268604  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.269913  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.130897 15826 consensus_queue.cc:1048] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:02:52.219858  8823 raft_consensus.cc:2468] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:52.230749  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:52.275702  8818 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
W20260709 13:02:52.275876  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45252.9us
I20260709 13:02:52.261083  8864 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:52.196703  8748 raft_consensus.cc:2468] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:52.278445  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.279489  8748 raft_consensus.cc:2410] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:52.247476 16256 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:52.244999  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 112982us
I20260709 13:02:52.172842 16179 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.229890 16312 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.280373  8741 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.212610 16136 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.285032  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.277278  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:52.298041  8864 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:52.299873  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:52.300711  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.293339 16179 raft_consensus.cc:3060] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.257719  8891 rpcz_store.cc:276] 0709 13:02:51.092501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.092624 (+   123us) service_pool.cc:224] Handling call
0709 13:02:51.092922 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.095133 (+  2211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.095161 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.693027 (+597866us) spinlock_profiling.cc:243] Waited 946 us on lock 0x7b2c00132978. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f4dc8a11 00007f90f4dd8372 00007f90f650aa01 00007f90f64cf43a 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2
0709 13:02:51.720175 (+ 27148us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.123326 (+403151us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:52.123377 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.123471 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.205200 (+ 81729us) raft_consensus.cc:1685] finished
0709 13:02:52.205221 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.205475 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18639,"raft.queue_time_us":37724,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":6334,"spinlock_wait_cycles":2176384,"thread_start_us":342082,"threads_started":1,"wal-append.queue_time_us":445744}
I20260709 13:02:52.246928 16138 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.289785 16174 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:52.298983  8745 raft_consensus.cc:2468] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:52.252859 16260 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.321306 16343 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:52.279793 16274 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:52.291368  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 142515us
I20260709 13:02:52.370921 16274 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:52.374681 16207 consensus_meta.cc:287] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.002s
I20260709 13:02:52.375172 16207 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.308305 16051 raft_consensus.cc:2804] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:52.376338 16051 raft_consensus.cc:697] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:52.376456  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.377795  8894 raft_consensus.cc:2410] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:52.258322 16302 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:52.349370 16321 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:52.412307  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.350103 16138 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.350907  8820 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:52.312398 16278 raft_consensus.cc:2804] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:52.355530  8746 raft_consensus.cc:1217] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 16.12->[17.13-20.15]   Dedup: 17.13->[18.14-20.15]
I20260709 13:02:52.419461  8746 pending_rounds.cc:85] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Aborting all ops after (but not including) 13
I20260709 13:02:52.419739  8746 pending_rounds.cc:107] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:02:52.419961  8746 raft_consensus.cc:2889] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.362062 16202 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.363675 16089 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.304312 16336 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:02:52.368214 16303 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.295241 16256 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.375461  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.377350 16051 consensus_queue.cc:237] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:52.353222  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.384220 16313 log.cc:927] Time spent T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.455s	user 0.002s	sys 0.002s
W20260709 13:02:52.384788 16322 log.cc:927] Time spent T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:02:52.385257 16135 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:52.315407 16175 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:52.387276 16331 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.193s	user 0.001s	sys 0.004s
W20260709 13:02:52.387351 15993 consensus_meta.cc:287] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.849s	user 0.008s	sys 0.003s
I20260709 13:02:52.305555 16136 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:52.326884  8741 raft_consensus.cc:2468] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 23.
I20260709 13:02:52.274360 16291 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.396188  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.403587 16279 raft_consensus.cc:3055] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:52.404763  9443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1443) took 1121 ms. Trace:
I20260709 13:02:52.450028  9443 rpcz_store.cc:276] 0709 13:02:51.283105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.300921 (+ 17816us) service_pool.cc:224] Handling call
0709 13:02:51.301232 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.567607 (+266375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.567639 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.567678 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:51.567721 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.404485 (+836764us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54005b09d4. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:52.404536 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.404676 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612219648}
I20260709 13:02:52.353408 16260 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.416920  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.417124  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.418195  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.418606  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.322840 16282 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.404644 16179 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.413957 16250 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:52.431710  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 154574us
I20260709 13:02:52.433311 16278 raft_consensus.cc:697] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:52.425489 16202 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.442515 16136 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.421942 16249 consensus_queue.cc:1048] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:02:52.441349 16135 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.462632  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.418671 16318 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.464771  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.460300 16250 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.458074 16263 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.449949 16279 raft_consensus.cc:740] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:52.473961  9414 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:52.446278 16175 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.483603  8741 raft_consensus.cc:2468] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:52.466790 16136 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:52.485169  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.488296  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.489058  8741 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:52.490103  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.490829  8741 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:52.491806  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.492460  8741 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.473529 16135 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:52.486510  8898 raft_consensus.cc:1240] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:52.507794  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0adf8736244d3b208cbc85700f0f9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:52.508572  9443 raft_consensus.cc:3060] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.510388 16256 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.495850  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:52.512053 16342 log.cc:927] Time spent T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:02:52.512677  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:52.466384 16273 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.481959 16250 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.516425  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.504964  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.517606  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.519539  8747 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:52.520884 16252 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:52.517984 15993 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:52.522967  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.517576  9447 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:52.531620  8747 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:52.517727 16344 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.466998 16202 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.533373  9443 raft_consensus.cc:2468] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:52.531953  8741 raft_consensus.cc:2468] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:02:52.539899  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:52.528213 15993 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.534437 16352 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:52.554322 16302 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.555814 16318 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.546686 16344 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.557611 16297 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:52.558386 16297 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.559553 16344 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.562170 16349 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.562696 16349 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.562988  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.564708 16349 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.565894 16352 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.566403 16352 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.564030  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.569926  8748 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 21: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.569646 16279 consensus_queue.cc:260] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.579082 16175 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.580415 16175 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.582196 16179 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.584259 16297 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.585053 16202 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.526052 16278 consensus_queue.cc:237] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.587301  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.588738  8742 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:52.534175 16303 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:52.590555  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 50750.7us
I20260709 13:02:52.596623  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.597453  8742 raft_consensus.cc:2393] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
W20260709 13:02:52.598227  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.607088  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.609206  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.536127 16207 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.580830 16175 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.628490  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.600456 16297 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:52.590253 15993 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.632620 16352 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.593530 16279 raft_consensus.cc:3060] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.649365  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:52.637996 16176 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:52.661358  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:52.661497  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.662992  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.663784  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.664696  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.665519  8823 raft_consensus.cc:2393] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 23: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:52.667855  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.668040  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.668489  8895 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:52.669970  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:52.670521  9447 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.641404 16297 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.671780  8816 raft_consensus.cc:2393] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:52.673435  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.674404  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.677389  8822 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.675263  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:52.684096  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46292.2us
I20260709 13:02:52.684912  8789 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:52.685813  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:52.689495  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 147876us
I20260709 13:02:52.680642 16297 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.654394 16265 consensus_queue.cc:1059] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:02:52.668493 16176 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.701882 16285 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:52.710574  8896 raft_consensus.cc:2468] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:02:52.696666  8789 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:52.712282  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.713765  8789 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:52.715606  8822 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
W20260709 13:02:52.707670  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 108379us
W20260709 13:02:52.720826  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 420188us
I20260709 13:02:52.725803  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:52.726805  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.709403 16318 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.714622 16258 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:52.732488 16318 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.745481 16357 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.746310 16357 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.748986  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.750028  8743 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 24: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.751029  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.731743 16312 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.753875 16175 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.755244 16352 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.735932 16356 consensus_queue.cc:1048] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:02:52.732115 16336 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:52.727556 16275 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:52.758641 16275 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.758980 16275 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.759908 16135 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.760562 16291 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.765938 16136 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.768085 16136 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.768649  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.769471  8744 raft_consensus.cc:2393] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.769655  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.770378  8895 raft_consensus.cc:2393] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:52.770215  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.770766  8744 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.771173  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.773665  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.774392  8892 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.775005  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.775537  8749 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.776823  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.778234  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:52.796463  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70724us
I20260709 13:02:52.797600  8789 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.801470  9444 raft_consensus.cc:1240] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:02:52.801931  9447 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
I20260709 13:02:52.806202  8789 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:52.812553  8789 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.779397 16312 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.815667  8744 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:52.817768  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.823035  8749 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:52.734134 16258 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.805224 16352 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.829834  8824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37554 (request call id 2718) took 1026 ms. Trace:
I20260709 13:02:52.831826  8789 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.834873  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.824889 16209 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:52.732715 16250 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.831768 16258 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:52.804035 16297 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.833640  8824 rpcz_store.cc:276] 0709 13:02:51.803851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.805086 (+  1235us) service_pool.cc:224] Handling call
0709 13:02:51.814669 (+  9583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.814902 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:02:52.228593 (+413691us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.817780 (+589187us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.829455 (+ 11675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:52.829504 (+    49us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:52.829546 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.829612 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.829758 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398782,"prepare.queue_time_us":168701,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":425774,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":18498,"thread_start_us":577741,"threads_started":3,"wal-append.queue_time_us":13508}
I20260709 13:02:52.850334  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.830593 16244 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.851524  8892 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
I20260709 13:02:52.855086  8744 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:52.837704 16257 raft_consensus.cc:2804] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:52.857662  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.862267  9443 raft_consensus.cc:2368] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18.
I20260709 13:02:52.863181  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.863934  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.864697  8823 raft_consensus.cc:3060] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.865839  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.867002  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:52.869246  8821 raft_consensus.cc:2393] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 24: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.869484  8743 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 23: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:52.871668  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.872238  8744 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:52.869728  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.871901  9415 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:52.876616  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.877103  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:52.873636 16355 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:52.877882  8823 raft_consensus.cc:2468] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:52.837754 16352 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.878938  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.845794 16275 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.868428 16096 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.870414 16349 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:52.893540 16135 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.893965 16276 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:52.908895  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.897372 16138 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:52.874555 16209 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:52.898811  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:52.914137  8747 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:52.915135  8747 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:02:52.893133  9415 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.880870 16202 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:52.894529 16320 log.cc:927] Time spent T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.638s	user 0.000s	sys 0.004s
I20260709 13:02:52.910185  8822 raft_consensus.cc:1240] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer 112105225cc140459688be5d31dcb071 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:52.911512  8744 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:52.890360 16257 raft_consensus.cc:697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:52.860761 16244 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.918423  9415 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:52.923629  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:52.919036  8744 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: [18.9-20.10]
I20260709 13:02:52.924289  8747 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:52.911510 16284 raft_consensus.cc:2804] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:52.882069 16355 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:52.939702 16374 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:52.944054  8889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50492 (request call id 3082) took 1300 ms. Trace:
W20260709 13:02:52.944702  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.928107 16363 raft_consensus.cc:2804] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:02:52.928560 16361 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:52.931804 16135 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:52.933506 16202 raft_consensus.cc:2804] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:52.935156 16244 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.906613 16096 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.914378 16330 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.956677 16330 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.957096  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:52.957821  8816 raft_consensus.cc:3060] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:52.957242 16357 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.949651 16276 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
W20260709 13:02:52.958626  8634 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet d4f573b86fda4b3e8fd982da9615dc8e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 29c7934ee528454785432732d01ed0a0 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:52.958870 16276 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:52.948372  8861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.954727  8893 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:52.913801 16209 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:52.961534 16209 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:52.944485  8889 rpcz_store.cc:276] 0709 13:02:51.644140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.646925 (+  2785us) service_pool.cc:224] Handling call
0709 13:02:51.647345 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.778612 (+131267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.778645 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.783380 (+  4735us) log.cc:844] Serialized 26 byte log entry
0709 13:02:51.931994 (+148614us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:51.967244 (+ 35250us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.014228 (+ 46984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.014310 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.817446 (+803136us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b2400044b28. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f32689c4 00007f90f3266acb 00007f90f3421dc9 00007f90f3421a55 00007f90f342109d 00007f90f605310e 00007f90f606a88c 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221
0709 13:02:52.943778 (+126332us) raft_consensus.cc:1685] finished
0709 13:02:52.943872 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.943998 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109854,"prepare.queue_time_us":42747,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":202931,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":22748,"spinlock_wait_cycles":14629120,"thread_start_us":235150,"threads_started":2,"wal-append.queue_time_us":313382}
I20260709 13:02:52.962402  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:52.963071  8743 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.963762 16096 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:52.963967  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.933590 16284 raft_consensus.cc:697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:52.964588  8889 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:52.965447 16284 consensus_queue.cc:237] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.966908 16275 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.967844  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.968509 16202 raft_consensus.cc:697] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:52.969460 16202 consensus_queue.cc:237] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.970158 16317 raft_consensus.cc:3055] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:52.970580 16317 raft_consensus.cc:740] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:52.971378 16317 consensus_queue.cc:260] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:52.971524  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:52.972430 16317 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:52.955909 16363 raft_consensus.cc:697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:52.952204 16297 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:52.920279 16349 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:52.974295  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:52.975291  8889 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:02:52.972082 15993 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.944007 16257 consensus_queue.cc:237] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.997694 16363 consensus_queue.cc:237] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:52.997566 15993 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:52.959265 16330 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.001232  8895 raft_consensus.cc:2393] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 24: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:53.009395  8817 raft_consensus.cc:1240] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:53.009763 16333 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.006s	sys 0.016s
W20260709 13:02:53.010102 16333 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.006s	sys 0.016s
I20260709 13:02:52.961097 16276 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:53.013019  8743 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:53.017572  8816 raft_consensus.cc:2468] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:53.019263  9415 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:53.020846 16202 raft_consensus.cc:2804] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:53.021514 16202 raft_consensus.cc:697] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:53.022599 16202 consensus_queue.cc:237] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.027331  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.030622  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.031523  8816 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.039453 16355 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.047389  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.053021 16377 consensus_queue.cc:1059] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:53.054345 16377 raft_consensus.cc:3055] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:53.054713 16377 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.055540 16377 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.056957 16377 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.061314  8862 leader_election.cc:400] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18.
I20260709 13:02:53.061761  8862 leader_election.cc:403] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:53.057360  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:53.065768  9414 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.066262 16209 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.067480 16357 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:53.067852 16323 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:53.069211  8816 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:53.072662  9415 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.081259  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:53.081634  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:53.069870 16250 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.067862 16357 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:53.095789 16357 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:53.091547 16276 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.098095 16276 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.101092 16269 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:53.101671 16269 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.079432 16135 raft_consensus.cc:2804] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:53.102649 16135 raft_consensus.cc:697] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:53.103725 16269 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.103581 16135 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.094980  8898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2912) took 1017 ms. Trace:
I20260709 13:02:53.104813  8898 rpcz_store.cc:276] 0709 13:02:52.078853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.079497 (+   644us) service_pool.cc:224] Handling call
0709 13:02:52.079757 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.507299 (+427542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.094750 (+587451us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400249354. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:53.094921 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935024512}
I20260709 13:02:53.109346 16276 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.109491  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.110092  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.110603  8863 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:53.111159  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.111272  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.113945  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.114024  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.114661  8816 raft_consensus.cc:2368] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18.
I20260709 13:02:53.114806  8823 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 16.
I20260709 13:02:53.116428  8816 raft_consensus.cc:1240] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:53.112556  8742 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:53.120651  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:53.120985  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:53.121259  8889 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:53.123474  8863 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:53.116930  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.125131  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.128482  8894 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:53.129249  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.130044  8816 raft_consensus.cc:3060] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.125654  8824 raft_consensus.cc:2393] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 21: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:53.122558  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:53.059192 16258 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.079638 15993 raft_consensus.cc:3060] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.137156  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.140540  8712 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.141561  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:53.141650  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.142795  8823 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:53.144105  8863 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.143149  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.147854  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.132638 16252 raft_consensus.cc:2804] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:53.167518 16251 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:53.212456 16251 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.212975 16251 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.167994 16344 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:53.217674 16344 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:53.170893 16258 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.171229 16319 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.171319 16260 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.140062 16244 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:53.171952  8821 raft_consensus.cc:1240] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:53.172915  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
W20260709 13:02:53.229548  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 56717.5us
I20260709 13:02:53.174165  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.174901 16125 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:53.175387  8824 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:53.180671 16317 consensus_queue.cc:1059] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:53.184043 16297 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.189756 16274 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:53.191015  8816 raft_consensus.cc:2468] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:02:53.136752 16269 consensus_queue.cc:1059] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:02:53.199040 16209 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:53.209913 16318 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.132889 16312 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:53.155241  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:53.226754  8823 raft_consensus.cc:1240] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:02:53.251067  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:53.232913  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 58836.9us
I20260709 13:02:53.197481 16282 raft_consensus.cc:2804] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:53.253898  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.255388  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.235249 16300 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:53.238912 16297 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.239182 16274 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:53.244395  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.153904 16279 consensus_queue.cc:1059] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:02:53.247042  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.218952 16344 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.228163 16252 raft_consensus.cc:697] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.237349 16202 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:53.259816 16324 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:02:53.261394  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 106244us
I20260709 13:02:53.232448 16319 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.258594  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:53.223623 16357 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.263221 16260 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.274724 16209 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:53.303735  8787 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:53.274544 16282 raft_consensus.cc:697] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:02:53.304045  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 45591.6us
I20260709 13:02:53.263162 16138 raft_consensus.cc:3055] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.274099 16274 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.262660 16312 raft_consensus.cc:697] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.303714 16323 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:53.296900 16265 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:53.334849  8715 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.275146  8864 leader_election.cc:400] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18.
I20260709 13:02:53.296864 16202 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:53.301636 16300 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.345942 16382 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:02:53.305460 16299 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:53.335530 16209 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:53.335817 16138 raft_consensus.cc:740] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:53.351908 16252 consensus_queue.cc:237] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:53.327363 16327 log.cc:927] Time spent T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 1.120s	user 0.004s	sys 0.006s
I20260709 13:02:53.360437  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.361171  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.375133 16330 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:53.387717  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 112770us
I20260709 13:02:53.390009  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.352224 16329 raft_consensus.cc:3055] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:53.392138  8740 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:02:53.354167 16257 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:53.399313  8746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.309054 16283 consensus_queue.cc:1059] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:02:53.384392 16125 raft_consensus.cc:3060] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.404011  8744 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:53.391072 16202 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:53.405270  8818 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:53.388477 16318 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.395696 16350 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.849s	user 0.006s	sys 0.000s
I20260709 13:02:53.414132  8789 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.395743 16209 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.412087  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:53.416591  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.417827 16363 raft_consensus.cc:3055] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:53.418257 16308 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:53.418282 16363 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:53.418883 16308 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.419302 16308 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.419386 16363 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.420708 16363 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.421171 16285 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:53.392270 16300 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.424326 16300 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.399020 16329 raft_consensus.cc:740] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.401713 16257 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.462491  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.463227  8743 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:53.410023 16244 consensus_queue.cc:1059] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:02:53.467975  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:53.469519  8891 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:53.473737 16288 raft_consensus.cc:3055] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:53.474221 16288 raft_consensus.cc:740] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.475054 16288 consensus_queue.cc:260] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.476084 16288 raft_consensus.cc:3060] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:53.412606 16318 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.478952 16352 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.479645 16318 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.481086 16274 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.483726 16274 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.484604  9412 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.484458 16329 consensus_queue.cc:260] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.485760 16329 raft_consensus.cc:3060] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.486105 16365 raft_consensus.cc:2804] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:53.460983 16138 consensus_queue.cc:260] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.486717 16365 raft_consensus.cc:697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:53.487099 16138 raft_consensus.cc:3060] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.487634 16365 consensus_queue.cc:237] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.486112 16251 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.492097 16251 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.411290 16282 consensus_queue.cc:237] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.494187  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.495323  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.496419  8749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.497123  8749 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:53.504516  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:53.505796  8749 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:53.467199 16312 consensus_queue.cc:237] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.498795 16297 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.465296 16257 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.509847 16244 raft_consensus.cc:3055] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:53.510179  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.502072  8741 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:53.512679  8740 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:02:53.514005  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.514946  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.515539  8824 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:53.516669  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.517058 16176 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:53.519392 16284 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:53.519627  8789 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.520511 16276 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:53.526146  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.526888  8862 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.528659 16136 consensus_queue.cc:1059] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:53.529593 16140 raft_consensus.cc:3055] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.529982 16140 raft_consensus.cc:740] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:53.530287  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.531049  8744 raft_consensus.cc:2393] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 20: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:53.530838 16140 consensus_queue.cc:260] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.531701 16297 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:53.532090 16297 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.532446 16297 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.532720 16319 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.510310 16244 raft_consensus.cc:740] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.533977 16244 consensus_queue.cc:260] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.534521  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:53.535351 16244 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:53.536032 16140 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.536229  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:53.536909  8889 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:53.502674  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
W20260709 13:02:53.470618 16350 log.cc:927] Time spent T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:53.537742  8743 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 17: Already voted for candidate b60591217ee24378bb66a86d6d0dda52 in this term.
I20260709 13:02:53.538683  8745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51668 (request call id 2846) took 1394 ms. Trace:
I20260709 13:02:53.538797  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.539106  8745 rpcz_store.cc:276] 0709 13:02:52.144679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.322320 (+177641us) service_pool.cc:224] Handling call
0709 13:02:52.322656 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.345172 (+ 22516us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:52.415357 (+ 70185us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.528102 (+112745us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:52.528518 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.528643 (+   125us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:52.535681 (+  7038us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.538926 (+  3245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.538995 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.960642 (+421647us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b180043e7c0. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2522 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:53.300605 (+339963us) spinlock_profiling.cc:243] Waited 874 us on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:53.538491 (+237886us) raft_consensus.cc:1685] finished
0709 13:02:53.538512 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.538613 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166273,"prepare.queue_time_us":88678,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":654,"raft.queue_time_us":72893,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":44835,"spinlock_wait_cycles":10150528,"thread_start_us":88332,"threads_started":1,"wal-append.queue_time_us":121734}
I20260709 13:02:53.539641 16319 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:53.541540 16319 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.541675 15993 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 17. Current term is 18.
I20260709 13:02:53.541929 16319 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.550827  9444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58306 (request call id 1448) took 1035 ms. Trace:
I20260709 13:02:53.551595 16125 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:53.551959 16387 log.cc:927] Time spent T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:02:53.551661 16319 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.552771  8896 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:53.553414  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.562505  8898 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 18.
I20260709 13:02:53.573555  8747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51680 (request call id 2617) took 1101 ms. Trace:
I20260709 13:02:53.561434  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.551273  9444 rpcz_store.cc:276] 0709 13:02:52.515201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.525370 (+ 10169us) service_pool.cc:224] Handling call
0709 13:02:52.525646 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.804699 (+279053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.550577 (+745878us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54004a4554. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cfe01 00007f90f64ce83e 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6
0709 13:02:53.550759 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588188544}
I20260709 13:02:53.586078  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.586496  8742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.589331  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.590090  8745 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:53.591925  8745 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:53.596153  8741 raft_consensus.cc:1240] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:02:53.597944  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.594185  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.608521  8891 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:53.614269  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:53.618654  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 108561us
I20260709 13:02:53.574066  8747 rpcz_store.cc:276] 0709 13:02:52.472009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.924932 (+452923us) service_pool.cc:224] Handling call
0709 13:02:52.937347 (+ 12415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.139220 (+201873us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:53.139617 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.394014 (+254397us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.394357 (+   343us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.394446 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:53.394464 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.394505 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.573276 (+178771us) raft_consensus.cc:1685] finished
0709 13:02:53.573319 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.573478 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210160,"prepare.queue_time_us":6146,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":202924,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":4103,"thread_start_us":253979,"threads_started":1,"wal-append.queue_time_us":349691}
I20260709 13:02:53.625629  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:53.618690  8749 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:53.641713  8744 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:02:53.641920  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.642339  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.648962  9443 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:53.658751  8741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.659626  8741 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.577965 16125 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:53.664608  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.643988 16249 raft_consensus.cc:2749] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:02:53.644009 16276 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:02:53.673388 15993 consensus_queue.cc:1059] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:53.674543 15993 raft_consensus.cc:3055] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:02:53.674909 15993 raft_consensus.cc:740] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.675704 15993 consensus_queue.cc:260] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.676941 15993 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:02:53.691035  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.649677  8788 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.691747  8746 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 21.
I20260709 13:02:53.677356 16369 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:53.699970  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.703447  8891 raft_consensus.cc:1275] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:53.702591  8744 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:53.640862 16138 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:53.715994  8893 raft_consensus.cc:1240] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
W20260709 13:02:53.702992 16390 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:53.727319  8895 raft_consensus.cc:1275] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:02:53.628640 16209 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:53.732264  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 82682.5us
I20260709 13:02:53.733361  8788 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.738878  8741 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:53.657918 16317 consensus_queue.cc:1059] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.079s
I20260709 13:02:53.649042 16261 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:02:53.727888 16252 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:53.789786  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.790477  8893 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.790180 16252 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.792603 16252 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:53.794030  8788 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.794911 16209 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.795710 16209 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:53.797873  8818 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 17.
I20260709 13:02:53.758946 16202 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.759743 16330 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:53.761013 16319 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.647718 16258 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:02:53.727804 16357 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.773608 16317 raft_consensus.cc:3055] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:53.777911 16297 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.802289  9444 raft_consensus.cc:1240] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:53.785584  8742 raft_consensus.cc:1240] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:53.752221 16138 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:53.812882  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 345807us
I20260709 13:02:53.813469  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.814400  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.815035  8889 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:53.816248  8896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2921) took 1031 ms. Trace:
I20260709 13:02:53.816566  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.822607  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.837555  8740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:53.839118  8789 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.829384  8787 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:53.838426  8818 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:53.841748  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.842332  8824 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.816644  8896 rpcz_store.cc:276] 0709 13:02:52.784419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.836115 (+ 51696us) service_pool.cc:224] Handling call
0709 13:02:53.552589 (+716474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.553697 (+  1108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.816058 (+262361us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b540046ee50. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64ce8ad 00007f90fb493419 00007f90f4a0abd4 00007f90f4a0ab62 00007f90f4a0aa91 00007f90f4a0aa0c 00007f90f4a09cc2 00007f90f480a48c 00007f90f48098c6 00007f90f480926f
0709 13:02:53.816188 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647042304}
I20260709 13:02:53.847344  8890 raft_consensus.cc:1240] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:53.846144  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:53.847985 15941 consensus_queue.cc:1048] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:02:53.768924 16257 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.803604 16258 raft_consensus.cc:3055] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:53.853741  8743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.854516  8743 raft_consensus.cc:2368] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:53.861673  8747 raft_consensus.cc:1275] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:53.801180 16202 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:53.866659  8863 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:53.869020  9413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.807135 16297 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:53.852746 16288 consensus_queue.cc:1059] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:53.804070 16317 raft_consensus.cc:740] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.838371 16266 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:53.886343  8822 raft_consensus.cc:1275] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:53.887987  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:53.894821  8820 raft_consensus.cc:3060] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.892637  8816 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:53.840003 16269 consensus_queue.cc:1048] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:02:53.855295 16370 raft_consensus.cc:2804] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:53.899602  8817 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:53.901662  8824 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 21.
I20260709 13:02:53.905094  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:53.845839 16244 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:53.865896 16258 raft_consensus.cc:740] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:53.803275 16308 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:53.910490  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:53.873828 16312 raft_consensus.cc:2804] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:53.905606  8715 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:53.853504 16261 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:02:53.923985  8820 raft_consensus.cc:2468] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:02:53.908134 16280 raft_consensus.cc:2764] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:02:53.936022  8789 leader_election.cc:400] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:53.936371  8789 leader_election.cc:403] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:02:53.945475  8863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.893563 16209 consensus_queue.cc:1059] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:02:53.935623 16288 consensus_queue.cc:1048] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:53.952894  8818 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 18.
I20260709 13:02:53.955024 16312 raft_consensus.cc:697] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:53.918629 16370 raft_consensus.cc:697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:53.916721 16125 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:53.965658  8894 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:53.977304  8747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:53.977949  8747 raft_consensus.cc:3060] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.973176  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:53.911300 16138 consensus_queue.cc:1059] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:53.964823  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:53.933444 16300 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:53.983374  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 374311us
I20260709 13:02:53.969798 16276 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991) with higher term. Message: Invalid argument: T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for earlier term 17. Current term is 18.
I20260709 13:02:53.948509 16363 consensus_queue.cc:1048] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:02:54.007723 16261 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.017314  8819 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:54.021425  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:54.018651  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.035327  8747 raft_consensus.cc:2468] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:53.969300 15826 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:02:54.041612  8815 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:02:53.999873 16138 consensus_queue.cc:1243] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer 7fd110c312c14e17a2bbe6d453493cf2 log is divergent from this leader: its last log entry 22.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:02:53.998646 16300 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:54.025619 16245 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:54.084173  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:54.085712  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 67168.8us
I20260709 13:02:54.094112 15941 raft_consensus.cc:3055] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:54.094702 15941 raft_consensus.cc:740] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:54.095746 15941 consensus_queue.cc:260] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.099023 16337 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:54.099777 16337 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:54.100858 16392 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.207s	user 0.000s	sys 0.002s
I20260709 13:02:54.088537 16312 consensus_queue.cc:237] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.089078 16378 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:54.106721 16378 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:53.969218 16317 consensus_queue.cc:260] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.082003 16279 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:54.110232 16279 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:54.088114 16388 log.cc:927] Time spent T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:54.110821 16318 raft_consensus.cc:3055] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:54.111238 16318 raft_consensus.cc:740] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:54.111677 16370 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:54.115516  8634 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet d4f573b86fda4b3e8fd982da9615dc8e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 29c7934ee528454785432732d01ed0a0 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:54.112524 16140 consensus_meta.cc:287] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.002s
I20260709 13:02:54.097160 15941 raft_consensus.cc:3060] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:54.140027  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 254959us
W20260709 13:02:54.143038  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.113272 16258 consensus_queue.cc:260] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.154485 16258 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.154681 16300 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.043707 16261 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.122208 16351 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:54.144887  8745 raft_consensus.cc:2468] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:02:54.161746  9444 raft_consensus.cc:1240] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:54.158010 16251 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:54.120086 16245 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.134881 16253 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
W20260709 13:02:54.184015 16365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.034s	sys 0.061s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 13:02:54.200191  9443 raft_consensus.cc:1275] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:54.202699  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
W20260709 13:02:54.188644 16365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.034s	sys 0.062s
I20260709 13:02:54.211519  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:54.212755  8748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
W20260709 13:02:54.209471  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 122208us
I20260709 13:02:54.218147  9414 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.219209  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.223145  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:02:54.224016  8820 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:54.229976  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.221869 16351 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:02:54.258836  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.225653 16352 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:54.219172 16378 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:54.209458 16406 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:54.233721 16337 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.222970 16136 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:02:54.221928 16051 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:54.325820 16402 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:54.327893  8745 raft_consensus.cc:1275] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:54.331362  8742 raft_consensus.cc:1240] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:02:54.335350 16261 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.315955 16363 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:02:54.336793  8744 raft_consensus.cc:1275] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:02:54.336890 16363 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:54.337561 16363 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:54.338436 16363 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.338780  8863 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:54.339625 16363 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:54.340710 16244 raft_consensus.cc:2804] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:54.341257 16244 raft_consensus.cc:697] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:54.341637  8789 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:54.342082 16244 consensus_queue.cc:237] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.342692 16308 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:54.345685 16369 raft_consensus.cc:2804] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:54.346186 16369 raft_consensus.cc:697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:54.346838  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52"
I20260709 13:02:54.357872  8744 raft_consensus.cc:2393] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fd110c312c14e17a2bbe6d453493cf2 in current term 19: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:54.319125  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 13:02:54.364331  8745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:54.370746  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:54.346992 16369 consensus_queue.cc:237] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:54.382508 16396 log.cc:927] Time spent T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 13:02:54.380488  8744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b60591217ee24378bb66a86d6d0dda52" is_pre_election: true
I20260709 13:02:54.443990  8744 raft_consensus.cc:2410] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:54.450809  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:54.453871  8822 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:02:54.464651  9414 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 32 similar messages]
I20260709 13:02:54.473585 16025 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.474300 16025 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.484416  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:54.488157  8822 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:54.397818 16312 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.402029  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.402271  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.410846 16251 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:54.417605  8820 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:02:54.419139 16397 consensus_queue.cc:1048] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:54.421974 16279 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:54.430030 16404 log.cc:927] Time spent T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:02:54.430532 16257 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.432891 16317 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:54.434034 16403 log.cc:927] Time spent T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:54.434190 16400 consensus_queue.cc:1048] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:02:54.435477 16245 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.358451 16051 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.318689 16269 consensus_queue.cc:1059] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.121s
I20260709 13:02:54.360365 15826 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:54.495016  8816 raft_consensus.cc:1240] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Rejecting Update request from peer 7fd110c312c14e17a2bbe6d453493cf2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:54.496749  8821 raft_consensus.cc:1275] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:02:54.533212  8821 pending_rounds.cc:85] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 10
I20260709 13:02:54.533545  8821 pending_rounds.cc:107] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:02:54.533811  8821 raft_consensus.cc:2889] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.534073  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:54.498481  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 102147us
I20260709 13:02:54.498592  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:54.535597  8824 raft_consensus.cc:2468] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:54.502180  8746 raft_consensus.cc:1275] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:54.424402 16318 consensus_queue.cc:260] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.504305  8740 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:54.476935 16025 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.515848 16176 raft_consensus.cc:493] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:54.522607  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.524335  8897 raft_consensus.cc:1275] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:54.525254  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f45eaa0e72045d5ae7ba8f95ca2ce75" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.509951 16251 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.521023 16284 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.539336  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.542330  8896 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
W20260709 13:02:54.530256  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 268062us
I20260709 13:02:54.554117  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:54.554935 16260 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:54.557936  9440 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:02:54.562146 16265 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:54.509505 16312 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.519531 16319 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:02:54.518573 16410 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:54.546614 16288 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:54.543013 15993 consensus_queue.cc:1048] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:02:54.586803 16051 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:54.546504 16176 raft_consensus.cc:515] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.588964 16176 leader_election.cc:290] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:54.590219 16411 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.173s	user 0.001s	sys 0.001s
I20260709 13:02:54.594614 16251 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:54.597159 16405 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.003s
I20260709 13:02:54.601528 16025 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:54.543192 15826 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.563270 16378 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:02:54.563117 16284 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:54.623486  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 158871us
W20260709 13:02:54.629242  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 117790us
W20260709 13:02:54.634155  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 154344us
I20260709 13:02:54.636150  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:54.637200 16297 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:54.638489 16324 raft_consensus.cc:2804] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:54.638939 16324 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.639285 16324 raft_consensus.cc:3060] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.624603 16276 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.632084 16408 log.cc:927] Time spent T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.225s	user 0.000s	sys 0.003s
I20260709 13:02:54.637742 16297 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.649199 16251 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:54.643569 16245 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:02:54.649809 16276 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.678191 16096 raft_consensus.cc:3055] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:54.681432 16378 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.678615 16413 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.649541 16318 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:54.715076  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:54.723064  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 161133us
I20260709 13:02:54.684958 16266 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:54.707317 16406 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:54.740255 16308 consensus_queue.cc:1048] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:54.744707  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:54.745807  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:54.717108 16096 raft_consensus.cc:740] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:54.748984  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:54.735803 16266 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.751699  8816 raft_consensus.cc:1275] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:54.761958 15826 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:54.764322  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:54.765475 16399 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:54.768074  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 228832us
I20260709 13:02:54.774163  8817 raft_consensus.cc:1275] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:02:54.710250 16282 consensus_queue.cc:1048] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:02:54.789813  8862 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.789872  8817 pending_rounds.cc:85] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 9
I20260709 13:02:54.792320  8817 pending_rounds.cc:107] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:02:54.792623  8817 raft_consensus.cc:2889] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.803443  8862 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:54.804737  8862 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:54.807842  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 43620.5us
I20260709 13:02:54.810866 16096 consensus_queue.cc:260] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.812170 16276 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.817276  8822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe13736604454be3b391b8e946192961"
dest_uuid: "29c7934ee528454785432732d01ed0a0"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52674
I20260709 13:02:54.817911  8822 raft_consensus.cc:606] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:54.818236  8822 raft_consensus.cc:614] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:54.821913 16284 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:54.806475 16176 consensus_queue.cc:1048] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:02:54.829305 16400 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:54.830013 16400 raft_consensus.cc:493] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:54.830544 16400 raft_consensus.cc:3060] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.834805 16265 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:54.836027 16282 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.837656 16406 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:54.839246  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.842736  8897 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:02:54.784073 16337 consensus_queue.cc:1048] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
W20260709 13:02:54.812525 16409 log.cc:927] Time spent T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.414s	user 0.001s	sys 0.002s
I20260709 13:02:54.847575  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:54.730449 16413 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.728108 16378 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.860158  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.791177 16370 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:54.861974  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.809449 15941 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.861150  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bb4c86bcc704ac2add360ac3e5823df" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:54.812345 16096 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.858346 16299 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:54.881731 16051 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:02:54.867942 16370 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.932505  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.875342 15941 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.941615  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.942593  8893 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:54.944094  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:54.939064  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 75619.5us
I20260709 13:02:54.945211  8898 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:02:54.929179 16297 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:54.947170  8862 leader_election.cc:304] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:54.953809  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:54.953233 16297 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:02:54.951737 16429 raft_consensus.cc:2749] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:55.001426 16421 log.cc:927] Time spent T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:02:55.002902 16436 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:55.005604 16433 log.cc:927] Time spent T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:02:55.012650 16432 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 13:02:55.015002 16312 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991)
I20260709 13:02:55.015697 15941 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:55.021962 16427 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:02:54.972421 16337 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.046610 16136 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.047358 16136 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:54.959556 16324 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.051357 16337 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.055641  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.056246  8891 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:55.056945 16337 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.060894  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.062857  8891 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:55.063427  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.070289  8820 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:02:55.076805  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 133372us
I20260709 13:02:55.049681 16344 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.058425 16300 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.100562  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.104003  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:55.170943  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.077090 16025 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.173583  8891 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:55.176739  8893 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer 29c7934ee528454785432732d01ed0a0 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:55.176522  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 58237.6us
I20260709 13:02:55.184124  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.184876  8893 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 24.
W20260709 13:02:55.182536  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.187947  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.188669  8893 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 25.
W20260709 13:02:55.190270 16438 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:02:55.116490 16413 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:55.194248  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.178292 16266 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.182153 16370 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.181367 16297 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.214326  8787 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:55.202977  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.186189 16025 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.177416 16300 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:55.221886 16444 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:02:55.116403 16344 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.186724 16319 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.228118  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.203974 16136 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:55.231675  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.276017  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 44421.1us
I20260709 13:02:55.194757 16324 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.240458 16176 consensus_queue.cc:1048] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:55.254259  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.256048  8823 raft_consensus.cc:1275] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 13:02:55.282353  8823 pending_rounds.cc:85] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Aborting all ops after (but not including) 11
I20260709 13:02:55.282637  8823 pending_rounds.cc:107] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:02:55.282903  8823 raft_consensus.cc:2889] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.255426  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:55.277590  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.236071 16300 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:55.284693  8789 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.272882  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44853.7us
I20260709 13:02:55.281425  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.255412 16370 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:55.297786  9449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.298730  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.298995  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.299793  9448 raft_consensus.cc:2393] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 25: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:55.278864 16138 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:55.306041  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:55.309693 16300 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:55.310999 16400 raft_consensus.cc:515] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.312536  9449 raft_consensus.cc:2410] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:02:55.313457  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:55.314369 16447 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:02:55.316154  8897 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.305233 16138 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.317230 16138 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:55.324604  8897 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:55.280148 16319 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.327715 16319 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.327580 16428 consensus_queue.cc:1048] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:55.297101 16355 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:55.332190  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.334438  8897 raft_consensus.cc:2393] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
W20260709 13:02:55.339861 16318 consensus_meta.cc:287] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.004s
I20260709 13:02:55.351172  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:55.296607 16442 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:55.231936 16297 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.378510  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.379523  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.402477  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.344630 16452 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.352818 16400 leader_election.cc:290] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.352039 16378 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.365515 16442 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.434551  8897 raft_consensus.cc:2468] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:02:55.418704 16442 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:55.424116 16452 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.447798 16452 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.449509 16319 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.450290 16319 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.451737 16025 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.453457 16297 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.455353 16319 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.457979 16344 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.459239 16442 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.462642 16458 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:55.492568 16442 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.508268 16297 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.508797 16319 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:55.508908 16297 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.509408 16319 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.511089 16297 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.511258 16319 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.513051  9412 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.514820  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.463541 16455 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.517522  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.517994  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.521118  9447 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:55.517966  8862 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:55.457115 16138 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.542536  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.474330 16344 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.543599  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 123143us
W20260709 13:02:55.546012  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.551554  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:55.542371 16455 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.551276 16442 raft_consensus.cc:2804] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:55.540020 16319 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:55.541695 16297 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.614373  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.614665  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:55.614991  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 510457us
W20260709 13:02:55.615320  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49035.2us
I20260709 13:02:55.621428  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.622011  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:55.623234  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:55.619109  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:55.619412  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.625669  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 79166.6us
W20260709 13:02:55.626286  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.630596  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:55.631325  8892 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:55.627502 16297 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.642133  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:55.663901  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:55.642866  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.667253  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.620590 16452 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.645038 16428 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:55.641269 16025 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:55.641003 16324 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.659464 16312 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.706120 16138 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.704933 16276 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.729642 16276 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.730103  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:55.730854 16463 log.cc:927] Time spent T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:55.731849 16276 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.732458 16297 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.732772 16324 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.714002 16452 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.737877 16452 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.738350 16312 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.740396 16312 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.710801 16399 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.741147 16399 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.743165 16399 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.661952 16250 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:55.744892 16250 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.730870  8821 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
I20260709 13:02:55.645025 16413 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.645331 16299 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.645485 16442 raft_consensus.cc:697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
W20260709 13:02:55.711186  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 422142us
I20260709 13:02:55.735026 16324 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.726133  8892 raft_consensus.cc:2468] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:02:55.663118 16406 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:02:55.759850  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.760499  8898 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 25.
I20260709 13:02:55.727120  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "234e3fa2a66f42f3a5a755f5f4adc858" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:55.761111 16176 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.761487  8816 raft_consensus.cc:2393] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 22: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:55.638784 16344 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.762885 16330 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.747171  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.628654 16319 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.754959 16297 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:55.755299 16284 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:55.756080 15941 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:55.752206  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.758220  9412 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.761649 16176 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:55.764108  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.765372  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:55.778059  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.778800  8821 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:02:55.761380 16406 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:55.753053 16455 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.752933 16260 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.798674 16260 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.801972  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.802769  8819 raft_consensus.cc:2393] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 in current term 23: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
I20260709 13:02:55.804018 16250 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.790673 16370 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:55.807566 16370 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:55.808001 16370 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:55.813493 16406 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.816169 16406 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.818884 16344 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.820628 16442 consensus_queue.cc:237] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.792457  9412 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:55.777697 16397 raft_consensus.cc:2804] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:55.763386 16330 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.772421 16125 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.800956 16260 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.827320  8892 raft_consensus.cc:1275] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:55.827255  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:55.832531  8788 leader_election.cc:304] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:55.833177  8896 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:55.801333  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.750774 16413 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.838881  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.836201  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:55.835309  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.841355  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 175085us
I20260709 13:02:55.842923  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.845639  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.821763 16400 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:55.854035  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.854534  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:55.855125  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.856020  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.856828  8863 leader_election.cc:336] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:55.857457  8863 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:55.858268 16429 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:55.858319  8789 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:55.830349 16369 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.860512  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:55.906975  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 44085.8us
W20260709 13:02:55.859602  8863 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:55.918673  8863 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:55.859620  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:55.919034  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 76605.5us
I20260709 13:02:55.861716  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.840317 16363 raft_consensus.cc:2749] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.839771 16441 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:02:55.927660  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 281185us
W20260709 13:02:55.929940  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 70370.5us
I20260709 13:02:55.829962 16244 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.922940  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 63367.8us
I20260709 13:02:55.844723 16397 raft_consensus.cc:697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 24 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:55.926720 16406 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:55.932427 16406 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.934541 16406 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.935039 16312 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:55.935062  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.935901 16269 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.936540  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.937269  8816 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:55.937338  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.938324  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.939018  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.929499 16458 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.939788  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.939746 16458 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.940416 16330 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.940721  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.941908 16458 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:55.936414 16269 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:55.944394  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:55.947682  8896 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
W20260709 13:02:55.948074  8789 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:55.948511  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:55.948984 16269 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.938565  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.949779  8817 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
I20260709 13:02:55.950631  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.951184  8817 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 23.
I20260709 13:02:55.951872 16413 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.932159 16397 consensus_queue.cc:237] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:55.940090  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.927524 16441 raft_consensus.cc:493] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.958096 16441 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:55.943841  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.964100  8820 raft_consensus.cc:2368] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:55.922523  8890 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:02:55.840245 16344 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.848399 16299 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.852545 16400 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.938314 16025 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:55.929572 16284 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.965900  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:55.946616 16444 log.cc:927] Time spent T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:55.929194 16369 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.988098  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:55.920192 16125 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:56.004266 16125 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.006676 16125 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.001148 16284 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.009521 16284 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:55.984781 16299 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.012351 16299 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.012506  8821 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:55.931496 16244 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.941834  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.001514 16391 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:56.014575 16391 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.015084 16391 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.019881 16406 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:56.020401 16176 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:56.020450 16406 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:56.021813 16472 log.cc:927] Time spent T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:55.985354 16400 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.022774 16406 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.023309 16406 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:56.023775 16406 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:55.945533 16337 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.999667  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 72073.7us
I20260709 13:02:56.031456  8892 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:02:56.037896  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.040644  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.042114  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.043316  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.043670  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.046344 16363 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:56.046538 16406 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.046859 16363 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:56.047850 16363 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.048460  8787 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:56.049458 16413 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:56.049962 16474 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:56.049453 16284 consensus_queue.cc:1048] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:56.050027 16413 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.050757 16474 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:56.051770 16474 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.052838 16299 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:56.053089  8787 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.053445 16299 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.054761  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.055757 16299 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.055800  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.056329 16458 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.056751  8898 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 19.
I20260709 13:02:56.056942 16458 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.057646  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.059304 16458 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.059763  8895 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:56.017403 16025 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.062109  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:56.045665  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.062340  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.064787  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.070878  8819 raft_consensus.cc:2393] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 20: Already voted for candidate 29c7934ee528454785432732d01ed0a0 in this term.
W20260709 13:02:56.079012  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.079197  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.086826  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:55.983891 16344 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.043596 16125 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:56.092516  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.098129  8891 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:56.093430  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:56.090862  9414 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.094421  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.095786  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.099898  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.065824 16413 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.067764 16406 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.099261  9445 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:02:56.081097 16330 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:56.108685  8789 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.060818 16337 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.117838  8643 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 7 to 9, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 9 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:56.112633  9414 leader_election.cc:336] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.089671 16260 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.112462 16330 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.093967 16319 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.120720  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.119875  9448 raft_consensus.cc:2393] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 24: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:56.119293  8863 leader_election.cc:336] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.151561  8863 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.143695  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.149776  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.097298 16244 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:56.113070 16475 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:56.165198  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.167770  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.169270  8713 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.182937 16479 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.184211  8713 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.184572  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.185493  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.150498 16330 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.187117  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:56.106077 16125 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.194352  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.155598 16458 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.153149 16429 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:56.195350  8789 leader_election.cc:336] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.199725  8789 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.114311 16406 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.201864  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:56.204564  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.150609 16260 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.212174  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.192829 16422 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:56.225229  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.228777  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.232193  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.207836 16476 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:56.238348  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.241683  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.243871  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.248781  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071
I20260709 13:02:56.251492  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.231388 16452 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.254621 16297 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:56.259898  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:56.246786  8695 auto_rebalancer.cc:893] Removing replace marker failed: Replica 29c7934ee528454785432732d01ed0a0 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:56.262862  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.205770 16458 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:56.266119  8695 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:02:56.265905  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:56.267055  8695 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:56.253922 16299 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:56.260286 16319 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:56.276176  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:56.279572  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.280679  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.268663 16448 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.268235 16422 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.297930 16260 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.301033 16025 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.301420 16299 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.282145 16319 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:56.279510 16297 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:56.266593 16452 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.279088 16369 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.293725 16479 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:56.314602 16319 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:56.337003 16406 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:56.338008  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.338634  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.314824 16344 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.338023 16479 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
W20260709 13:02:56.345892  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65336.9us
I20260709 13:02:56.353578  8715 leader_election.cc:400] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:56.353899  8715 leader_election.cc:403] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:56.355016  8715 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.355283  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.357154  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:56.347754 16370 consensus_meta.cc:287] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 13:02:56.358918  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.368374  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.368912  8715 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.350239 16250 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.353401 16400 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.376677 16250 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.377787  9447 raft_consensus.cc:2368] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21.
I20260709 13:02:56.356269 16479 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.375475  8890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.375926  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.358210 16344 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.380664 16467 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:56.367573 16324 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:56.377099 16025 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:56.385825  8712 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:56.386911 16300 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:56.387559 16300 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.387909 16300 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.388100 16458 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:56.389518  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.396674  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.365054 16125 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.398554  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.400848  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.402412  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.402992  8816 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 18.
I20260709 13:02:56.403036  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.367342 16370 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.378459 16406 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.406424  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.404091 16497 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:56.365100 16330 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.408665 16458 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:56.397725 16448 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.400031 16467 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:56.416653  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:56.423628 16344 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:56.427424 16452 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.430090 16441 raft_consensus.cc:515] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.431164  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:56.431945  8894 raft_consensus.cc:2393] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 24: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
W20260709 13:02:56.433687  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.407748 16125 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.438022  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:56.439174  8816 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:02:56.439548  8789 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.440016  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.441040  8819 raft_consensus.cc:2368] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:56.443980 16250 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:56.434549  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:56.445156  8863 leader_election.cc:336] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.446159  8863 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.404963 16319 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.447183 16441 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.447767 16337 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:56.449193 16330 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.449402 16400 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:56.450289 16319 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.450363 16300 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.453449 16300 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.455029 16363 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.457630  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.458807  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:56.455488  8789 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.456182 16096 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.462128  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:56.461270  8789 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.463292  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:56.463433  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.463840  8891 raft_consensus.cc:3060] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.463879  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:56.465621  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.465646  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
W20260709 13:02:56.467710  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:56.468225  9414 leader_election.cc:336] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:02:56.469789  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:02:56.470736  8863 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.471139  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:56.474334  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:56.476404 16317 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.441248 15826 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:56.478803  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:56.478106  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:56.480942  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:56.482758  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:02:56.483625  8863 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.484077  8863 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.484743  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.480307 15826 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.485538 16300 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.501972  8816 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 26.
I20260709 13:02:56.492563  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.493777 16319 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:02:56.460486 16391 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.515416  8894 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:56.517669  8713 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.521245  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.522332  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:56.583604  8817 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:56.583669  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 66100us
I20260709 13:02:56.522742 16370 leader_election.cc:290] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 election: Requested vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.523540  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:56.524649  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:56.490248 16496 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:56.529870  8632 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet d4f573b86fda4b3e8fd982da9615dc8e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:02:56.592671  8632 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62993.2us
I20260709 13:02:56.540474  8822 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:56.505673 15826 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.590293  8713 leader_election.cc:400] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21.
I20260709 13:02:56.596580  8713 leader_election.cc:403] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:56.558965  8818 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 22.
I20260709 13:02:56.556972  8643 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 6 to 11, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 11 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:56.565649  8821 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:56.569403  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:02:56.572041  9415 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:56.573472  8891 raft_consensus.cc:2468] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 20.
I20260709 13:02:56.577574  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
W20260709 13:02:56.581660  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 89188.3us
I20260709 13:02:56.585136  8819 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:02:56.556875 16479 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.590253 16337 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.582244 16467 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:56.591301 16319 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:56.626952  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:56.627854  8819 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.601450  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.610812  8892 raft_consensus.cc:2468] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 25.
I20260709 13:02:56.603749  8643 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 10 to 11, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 11 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:56.619335  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 50074.2us
I20260709 13:02:56.611496  8644 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 288) took 10090 ms. Trace:
I20260709 13:02:56.618639  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:56.635594  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:56.660539  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.667721  8715 leader_election.cc:400] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20.
I20260709 13:02:56.630659  8644 rpcz_store.cc:276] 0709 13:02:46.505607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.506041 (+   434us) service_pool.cc:224] Handling call
0709 13:02:56.595329 (+10089288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:51.509943 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:51.529581 (+ 19638us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:51.529636 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:02:51.529658 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:51.626370 (+ 96712us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:51.626494 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:51.626521 (+    27us) tablet.cc:685] Acquiring locks for 30 operations
0709 13:02:51.628641 (+  2120us) tablet.cc:701] Row locks acquired
0709 13:02:51.628691 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:02:51.629055 (+   364us) write_op.cc:270] Start()
0709 13:02:51.629215 (+   160us) write_op.cc:276] Timestamp: P: 1783602171628785 usec, L: 0
0709 13:02:51.629237 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:02:51.630067 (+   830us) log.cc:844] Serialized 11898 byte log entry
0709 13:02:52.345861 (+715794us) op_driver.cc:464] REPLICATION: finished
0709 13:02:52.975667 (+629806us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f9f05ef0
0709 13:02:53.030712 (+ 55045us) write_op.cc:301] APPLY: starting
0709 13:02:53.030832 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:53.524275 (+493443us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b1800363220. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33e2557 00007f90f33e47c1 00007f90f33e4779 00007f90f33e4709 00007f90f33e46d1 00007f90f33e39cd 000000000038f354 000000000038f199 00007f90f33c583a
0709 13:02:53.526977 (+  2702us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b5400385c54. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f6438571 00007f90f64cc32a 00007f90f6451f14 00007f90f6451ec3 00007f90f6451e38 00007f90f6451df3 00007f90f64510dc 00007f90f647ae16 00007f90f6462cbb 00007f90f644d39d 00007f90f6450d75
0709 13:02:53.770846 (+243869us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:53.770883 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:54.432244 (+661361us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:54.432584 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 13:02:54.432628 (+    44us) write_op.cc:312] APPLY: finished
0709 13:02:54.433131 (+   503us) log.cc:844] Serialized 260 byte log entry
0709 13:02:54.479381 (+ 46250us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f33c50ac 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b
0709 13:02:54.765556 (+286175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.765800 (+   244us) write_op.cc:454] Released schema lock
0709 13:02:54.765959 (+   159us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:54.803874 (+ 37915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:54.806947 (+  3073us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":436603,"reported_tablets":48,"spinlock_wait_cycles":594304,"child_traces":[["op",{"apply.queue_time_us":638131,"apply.run_cpu_time_us":65613,"apply.run_wall_time_us":1749456,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129467,"num_ops":30,"prepare.queue_time_us":287894,"prepare.run_cpu_time_us":69116,"prepare.run_wall_time_us":290294,"raft.queue_time_us":240342,"raft.run_cpu_time_us":26699,"raft.run_wall_time_us":1226581,"replication_time_us":683901,"spinlock_wait_cycles":134185728,"thread_start_us":1385914,"threads_started":5,"wal-append.queue_time_us":815656,"wal-append.run_cpu_time_us":15721,"wal-append.run_wall_time_us":2356434}]]}
I20260709 13:02:56.603143 16244 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:56.618093 16344 raft_consensus.cc:3060] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.648172 16442 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:56.665930 16397 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:56.628819 16245 raft_consensus.cc:2804] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:02:56.668884  8715 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:56.667212 16319 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.656127  8817 raft_consensus.cc:2468] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:02:56.625782 16337 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.644701 16496 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.717095  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:56.688452 16442 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.704923 16397 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.749035 16319 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:56.710088 16467 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:56.713406 16245 raft_consensus.cc:697] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 26 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:56.761868 16442 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:56.765755 16397 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:56.767445 16319 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:56.784931 16467 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 22 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:56.761193 16370 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:02:56.804358 16319 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.804430 16136 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:56.828203 16479 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.834470 16484 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.205s
I20260709 13:02:56.837319 16467 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.734500  8787 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:56.885231  8819 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:02:56.885581  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:56.827168 16136 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.895984  8864 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:02:56.897941  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 248218us
I20260709 13:02:56.913599 16318 raft_consensus.cc:2804] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:56.914433 16318 raft_consensus.cc:697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:56.916541 16136 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.916258 16318 consensus_queue.cc:237] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.924404 16317 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.925113 16337 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.907184 16330 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:56.921416 16324 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.919142 16413 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:56.947455 16265 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:56.825464 16245 consensus_queue.cc:237] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.953140 16413 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.942473 16136 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:56.938977 16324 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:56.950240 16265 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.962113 16324 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:56.962813 16324 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.963584 16324 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:56.964447  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 289613us
I20260709 13:02:56.966323 16136 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:56.974179 16363 raft_consensus.cc:2804] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:02:56.965489  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:56.993697  9415 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:56.995388 16352 raft_consensus.cc:2804] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:56.996079 16352 raft_consensus.cc:697] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:56.997243 16391 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:56.999259  9414 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:56.999115 16352 consensus_queue.cc:237] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.007584 16363 raft_consensus.cc:697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:02:57.009253 16363 consensus_queue.cc:237] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.012846 16344 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 20. Current term is 21.
I20260709 13:02:57.018764 16413 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:57.033699  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46461.1us
W20260709 13:02:57.038507  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 73201.1us
W20260709 13:02:57.039744  9414 leader_election.cc:336] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.041590 16135 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:57.050814 16506 log.cc:927] Time spent T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:02:57.052405 16509 log.cc:927] Time spent T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.210s	user 0.000s	sys 0.004s
W20260709 13:02:57.057083 16502 log.cc:927] Time spent T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.218s	user 0.002s	sys 0.000s
I20260709 13:02:57.021991 16265 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.070452 15826 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:57.053186 16503 log.cc:927] Time spent T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
W20260709 13:02:57.052608 16505 log.cc:927] Time spent T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.212s	user 0.000s	sys 0.002s
I20260709 13:02:57.049054  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.085693  8818 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 24.
I20260709 13:02:57.081676 16324 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.096387  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
W20260709 13:02:57.095387  8788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.111790  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:57.114279 16504 log.cc:927] Time spent T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.279s	user 0.000s	sys 0.003s
I20260709 13:02:57.114073 16397 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.122854  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.072752  8643 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 8 to 9, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 9 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:57.134274  8818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:57.131281  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 326733us
I20260709 13:02:57.147549  8892 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:57.150161  8643 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:57.157804  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.158075  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.162493  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.165884  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.166651  8787 leader_election.cc:304] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:57.168200  9440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:57.169250  9440 raft_consensus.cc:2393] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 22: Already voted for candidate 112105225cc140459688be5d31dcb071 in this term.
I20260709 13:02:57.170369  8897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50482 (request call id 2972) took 1242 ms. Trace:
I20260709 13:02:57.170818  8897 rpcz_store.cc:276] 0709 13:02:55.928006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.928159 (+   153us) service_pool.cc:224] Handling call
0709 13:02:55.928452 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.928772 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.928800 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.929239 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.519632 (+590393us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800342040. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90fc2aade1 00007f90fc2b75b1 00007f90f33c4e07 00007f90fc21fab9 00007f90f33e169b 00007f90f33df93f 00007f90f33e1c02 00007f90f606920b 00007f90f606ed23 00007f90f606f0d1 00007f90f648aa0f
0709 13:02:56.922254 (+402622us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:56.922342 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.923255 (+   913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.052166 (+128911us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b4400022b4c. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f3308c85 00007f90f33096a2 00007f90f607b1db 00007f90f6076aaf 00007f90f606a86e 00007f90f6069bb6 00007f90f60697d8 00007f90f6073221 00007f90f60731d9 00007f90f6073169
0709 13:02:57.169961 (+117795us) raft_consensus.cc:1685] finished
0709 13:02:57.169994 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.170177 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1003,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":227211904,"thread_start_us":401457,"threads_started":1,"wal-append.queue_time_us":901645}
I20260709 13:02:57.173411  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.173576 16136 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.179912 16512 raft_consensus.cc:2764] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 13:02:57.181505 16245 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:57.186360  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.184850  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.185720  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.186964 16245 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.190603  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.191665 16245 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:57.193647  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46237.7us
I20260709 13:02:57.192535  8713 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:57.148267 16319 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.202586  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.207634  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:57.208786  9442 raft_consensus.cc:2368] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 24. Current term is 25.
I20260709 13:02:57.214592 16442 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.235203  9447 raft_consensus.cc:3060] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:57.235765  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "410ca1c9d2fd42c68bdf6bb996508fd9" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:57.242059  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 49726.6us
I20260709 13:02:57.242187 16300 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:57.244688  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.244808 16452 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:57.245810  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.245707 16452 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.248220 16452 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.223155 16397 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:57.225245  8894 raft_consensus.cc:1275] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:02:57.199461 16344 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:57.226589 16391 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:57.226894 16319 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.229102  8715 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:57.256186 16391 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:57.257212  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 80656.9us
I20260709 13:02:57.230602  8892 raft_consensus.cc:1275] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:57.258773 16391 leader_election.cc:290] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.233062  8891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.273984  9447 raft_consensus.cc:2468] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
W20260709 13:02:57.310204  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 49396.1us
I20260709 13:02:57.317166  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:57.336790  8822 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
W20260709 13:02:57.269511  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:57.353629  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84205.9us
I20260709 13:02:57.260958 16299 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:57.273097 15826 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:57.392177  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62331.2us
I20260709 13:02:57.397042  9447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dada29d2380a453682f5dfef36dcf407" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:02:57.379774 16299 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:57.375875 16344 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.398366 16442 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.415736  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.420989  8892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.420768  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:57.430661  9441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:02:57.436074  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 169136us
I20260709 13:02:57.437369 16496 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:57.438297  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.444839  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:57.451292  8713 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:57.456148  8789 leader_election.cc:336] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.460932  8789 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:02:57.465996 16516 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:02:57.469843 16299 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.420506 16344 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.479710  8715 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:57.480523 16363 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:57.484648  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 58401.3us
I20260709 13:02:57.494519 16135 consensus_queue.cc:1048] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:02:57.495152  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 64561.5us
I20260709 13:02:57.495779 16344 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.496160 16135 consensus_queue.cc:1048] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:02:57.500962  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:57.510247  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 72913.6us
I20260709 13:02:57.454865 16413 consensus_queue.cc:1059] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:57.521461  8712 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:57.520529  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 90229.7us
I20260709 13:02:57.522019  8862 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:57.528957  8713 leader_election.cc:400] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 election: Vote denied by peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693) with higher term. Message: Invalid argument: T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for earlier term 24. Current term is 25.
I20260709 13:02:57.529980  8712 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.531773  8819 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:02:57.512020 16344 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:57.534461  8712 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.513041 16265 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.551875 16319 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.552685 16319 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:02:57.557158  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 56951.3us
I20260709 13:02:57.559813 16467 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:02:57.560731 16467 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.540588 16318 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.542114 16344 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:57.565490 16265 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.581185  8713 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.610420 16176 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:02:57.604161  9415 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.617264  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "739742728baa463f9d9f5e5884dd2e30" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:57.622301  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 87148.9us
I20260709 13:02:57.626021 16349 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:57.626883  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.634056  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:57.634718  8822 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:57.644325  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:57.645202  8897 raft_consensus.cc:2393] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:57.654927  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.664291  8890 raft_consensus.cc:3060] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:57.664459  8822 raft_consensus.cc:2468] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 25.
W20260709 13:02:57.669399  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 65362.7us
W20260709 13:02:57.672617  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 109997us
I20260709 13:02:57.677762  9440 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:57.679836  8819 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:57.656544 16265 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:57.690688 16413 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:57.693882  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "932d7bc525ba4af99f7d6b52fc1d7337" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:57.694661  8820 raft_consensus.cc:2368] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24.
I20260709 13:02:57.623684 16299 raft_consensus.cc:3060] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:57.627624 16344 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.709254  8895 raft_consensus.cc:1275] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:57.681708  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.723543 16363 consensus_queue.cc:1048] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:57.726552  9448 raft_consensus.cc:1275] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:02:57.729620  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.730301  8816 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
W20260709 13:02:57.731161  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 416909us
I20260709 13:02:57.732352  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:02:57.734790 15826 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:57.735348 15826 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.735488  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:57.735832 15826 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:57.736677 16479 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:57.739256  8817 raft_consensus.cc:1275] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:57.742683  9415 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.744644  9415 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:57.745494 16135 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.713137  9442 consensus_queue.cc:237] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.751089 16324 raft_consensus.cc:3055] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:57.751585 16324 raft_consensus.cc:740] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
W20260709 13:02:57.766260  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 46631us
I20260709 13:02:57.768267 16300 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.768338  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:57.769428 16530 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.770053 16378 consensus_queue.cc:1048] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:57.768942 16300 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.773531 16300 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.775411 16299 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:57.742959  8817 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 26 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:02:57.752535 16324 consensus_queue.cc:260] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.779896  8890 raft_consensus.cc:1275] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:02:57.781880  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.777796  9415 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:02:57.789212 16324 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.802711  9415 leader_election.cc:400] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24.
I20260709 13:02:57.803344  9415 leader_election.cc:403] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.784575 16135 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:57.809242  8863 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.817947  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.818400 16355 raft_consensus.cc:2804] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:02:57.819021 16355 raft_consensus.cc:697] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:02:57.820196 16355 consensus_queue.cc:237] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.821229 16349 consensus_queue.cc:1048] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:57.840644  8715 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
W20260709 13:02:57.843071  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 62557.8us
W20260709 13:02:57.844164  8789 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.851536 16299 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.849468 16300 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.867404  8789 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:57.916863  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.918385 16479 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.919108 16479 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.922137 16479 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.923815  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:57.929176  8821 raft_consensus.cc:1275] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:57.873111 16441 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.125s
W20260709 13:02:57.931710  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.932135  8863 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:57.861388 16300 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:57.955806 16535 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:57.956215  8863 leader_election.cc:336] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.956758  8863 leader_election.cc:304] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:57.952850 16479 raft_consensus.cc:493] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:57.960880 16479 raft_consensus.cc:515] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:57.961787 16535 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:57.962401  8863 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:57.963666 16479 leader_election.cc:290] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:57.965616  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:57.981576 16300 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:57.982352 16507 consensus_queue.cc:1048] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 13:02:57.984391  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:57.985230  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:58.001945 16300 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.002802 16300 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.008769  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.019536 15826 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:58.026140  8863 leader_election.cc:336] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:58.026386 16479 raft_consensus.cc:493] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:58.027259 16479 raft_consensus.cc:515] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.033092  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.036402 16324 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.037086 16324 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.041671 16300 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.068661  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:58.069676  8821 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.099920  8712 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:58.111563  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 228014us
I20260709 13:02:58.062717 15826 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:58.135952  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.097182 16397 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) with higher term. Message: Invalid argument: T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 112105225cc140459688be5d31dcb071 for earlier term 23. Current term is 24.
I20260709 13:02:58.171497  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6dcf2342014ed6b65dd5af29d18d3b" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.133841 16530 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.174633  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.166592 16452 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.178645  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:58.198330  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 233077us
I20260709 13:02:58.170130 16496 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.179049 16479 leader_election.cc:290] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.219310  9448 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 LEADER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 24. Current term is 25. Ops: []
I20260709 13:02:58.222424  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.185631 16324 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:58.225523  8712 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 54906.5us
I20260709 13:02:58.198016 16525 raft_consensus.cc:493] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
W20260709 13:02:58.230705  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 57238.2us
I20260709 13:02:58.230979  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.231252  8893 raft_consensus.cc:1240] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:58.207206 16452 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.218950 16496 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.274309 16529 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.278327 16529 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:02:58.274082  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 70553us
I20260709 13:02:58.280658 16529 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.286085 16496 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.286949  8713 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:58.249540 16525 raft_consensus.cc:515] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.296397 16525 leader_election.cc:290] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:58.311056  8715 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.256086 16300 raft_consensus.cc:493] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.318310 16452 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.325899 16300 raft_consensus.cc:515] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.328619 16300 leader_election.cc:290] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:58.335908 16542 log.cc:927] Time spent T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:02:58.338927 16550 log.cc:927] Time spent T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:58.340179 16496 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:58.333125 16540 log.cc:927] Time spent T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Append to log took a long time: real 0.102s	user 0.002s	sys 0.001s
W20260709 13:02:58.357940  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 48127us
I20260709 13:02:58.361977  8821 raft_consensus.cc:2468] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
W20260709 13:02:58.367082  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 65401.6us
I20260709 13:02:58.369823  8896 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:58.373772  8712 leader_election.cc:304] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.374313  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.370906 16300 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:58.382592  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.383018  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.384814  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.383402  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d038e54cdc4dfaa0cfec37a43480ee" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.391312  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.392228  8712 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.396157  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:58.402546  9414 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:58.404474  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:58.406590  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.397152 16529 consensus_queue.cc:1059] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:02:58.412778 16526 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:58.413877 16452 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.383344 16300 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.390172 16496 raft_consensus.cc:2749] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.412967 16529 consensus_queue.cc:1243] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer 112105225cc140459688be5d31dcb071 log is divergent from this leader: its last log entry 25.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:58.406917 16479 raft_consensus.cc:2749] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.414547 16452 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.405244 16525 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:58.424957 16452 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.412623 16299 raft_consensus.cc:493] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.416337 16300 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.438372  8712 leader_election.cc:304] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.439738  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.431841 16452 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.441151  8864 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:58.425001 16525 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.436510 16560 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.449239  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.446817 16344 raft_consensus.cc:2749] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:58.470243  9412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.455142 16265 raft_consensus.cc:2804] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:58.435530 16299 raft_consensus.cc:515] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.444511 16300 consensus_queue.cc:1059] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:02:58.493988 16300 raft_consensus.cc:3055] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:58.494575 16300 raft_consensus.cc:740] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:58.495407 16299 leader_election.cc:290] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.462771 16525 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.498189 16525 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.487855 16555 consensus_queue.cc:1048] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:02:58.511053 16479 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.511838 16479 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:58.512383  9414 leader_election.cc:336] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:58.515488 16265 raft_consensus.cc:697] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:02:58.452564 16452 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.519816 16452 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:58.531041  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:58.510869 16299 raft_consensus.cc:493] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:58.534139  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.548350 16557 raft_consensus.cc:3055] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:02:58.548974 16557 raft_consensus.cc:740] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:58.550068 16557 consensus_queue.cc:260] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.551769 16557 raft_consensus.cc:3060] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:58.556928  8819 raft_consensus.cc:1240] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 24. Current term is 25. Ops: []
W20260709 13:02:58.582754  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 109485us
I20260709 13:02:58.516814 16265 consensus_queue.cc:237] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.544379 16299 raft_consensus.cc:515] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.601912  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb392189d54431c863a3b787fb95779" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.604107 16299 leader_election.cc:290] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.608366 16479 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:58.614069  8713 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.615137 16452 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:58.615654 16452 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.599261 16530 raft_consensus.cc:493] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:58.624011  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac82cfba1d244139d9b06d8a6c88635" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.630007  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.630733  8823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.632128  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f091122e10604f27acd758d1f8da8081" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.633406  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.634678  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bf6ec0e6a745d0967928a3361eb8d8" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.637897 16565 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.638470 16565 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.639081 16561 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.639628 16561 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.640755 16565 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.495574 16300 consensus_queue.cc:260] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.643405  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.643792 16300 raft_consensus.cc:3060] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.644515  8824 raft_consensus.cc:1240] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Rejecting Update request from peer b60591217ee24378bb66a86d6d0dda52 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:58.646365  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.647048  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.647365 16561 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.647854  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.623781 16530 raft_consensus.cc:515] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:58.622675  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 50164.8us
W20260709 13:02:58.650695  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:58.667896  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:58.667333  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 211422us
I20260709 13:02:58.673220  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.672899  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.677990  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a7b40e33c24527acba97c95cee9272" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.689313  8819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24c6743859d43109fabeaf7ee8c8149" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.682123  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "265e5f9784a6426482f5948fbfcc995c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.719919  8816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.694758 16525 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.732152 16562 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.732985 16562 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.736913 16562 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:02:58.740998  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:58.714257 16479 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:58.747145  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.748315 16526 raft_consensus.cc:493] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:02:58.748940 16526 raft_consensus.cc:515] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.751839 16526 leader_election.cc:290] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.752328 16526 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.754710  8715 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:58.757164  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.757750  8817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.758936  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.760954  8715 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.763803  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.756676 16526 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.765436 16413 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:58.765537  8894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de733f99bf86451da507373e7f4b78aa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.764456  8889 raft_consensus.cc:2468] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 18.
I20260709 13:02:58.774433 16529 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:58.774765  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.775692 16529 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.776021  8715 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.738008 16452 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.778363 16452 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.778576  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.777176  8715 leader_election.cc:304] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.783452 16496 raft_consensus.cc:2749] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.784420  8715 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.785811 16529 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:58.787485  8715 leader_election.cc:304] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:58.766745 16526 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.789278 16530 leader_election.cc:290] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.789889 16526 raft_consensus.cc:2749] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:58.761364 16479 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:58.792550  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.793372 16526 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:58.794679  8715 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.795423 16526 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:58.796445  8715 leader_election.cc:304] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:58.797643  8715 leader_election.cc:304] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.798688  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.806592  8822 raft_consensus.cc:1275] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:02:58.811354 16496 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:58.812698  8715 leader_election.cc:304] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.807307  8897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.814272  8715 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:58.816840  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.820855  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.730394 16525 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.827883  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ea7c9c3095497eb93f5d5d412ad667" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.812453 16530 raft_consensus.cc:2749] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.804728 16299 raft_consensus.cc:2749] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:58.816821 16413 raft_consensus.cc:2749] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.835379 16555 consensus_queue.cc:1048] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:02:58.840282 16526 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:58.846906  8695 auto_rebalancer.cc:629] Failed to schedule move for tablet cb3dbb5cac024c52a371cbe4f0f1e952: Illegal state: Replica 29c7934ee528454785432732d01ed0a0 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:58.829648 16557 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.850711 16479 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:58.862210  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:58.868956 16569 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:02:58.869737 16569 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.872706 16569 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.881734 16525 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.895712  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.896714  8896 raft_consensus.cc:2393] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 29c7934ee528454785432732d01ed0a0 in current term 19: Already voted for candidate 7fd110c312c14e17a2bbe6d453493cf2 in this term.
I20260709 13:02:58.913730 16559 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index 3 to 11, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:58.915323 16496 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.915868 16496 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:58.917953 16496 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:58.925002  8821 raft_consensus.cc:2955] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index 3 to 11, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:58.927294  8820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.934672  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.937676  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:58.939216  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:58.942487 16578 log.cc:927] Time spent T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: Append to log took a long time: real 0.131s	user 0.003s	sys 0.000s
I20260709 13:02:58.947038  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:58.949735 16299 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:58.964354 16559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58344 (request call id 1) took 1289 ms. Trace:
I20260709 13:02:58.971378 16559 rpcz_store.cc:276] 0709 13:02:57.671915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.710595 (+ 38680us) service_pool.cc:224] Handling call
0709 13:02:57.748366 (+ 37771us) log.cc:844] Serialized 501 byte log entry
0709 13:02:58.961712 (+1213346us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":551,"raft.run_cpu_time_us":1260,"raft.run_wall_time_us":1258}
I20260709 13:02:58.973023  8632 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d4f573b86fda4b3e8fd982da9615dc8e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
W20260709 13:02:58.984027  8643 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 302) took 80246 ms (client timeout 15000 ms). Trace:
W20260709 13:02:58.984453  8643 rpcz_store.cc:269] 0709 13:01:38.737359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.738467 (+  1108us) service_pool.cc:224] Handling call
0709 13:02:58.983923 (+80245456us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:57.551990 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:57.552721 (+   731us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:57.552783 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:02:57.552816 (+    33us) tablet.cc:662] Decoding operations
0709 13:02:57.553589 (+   773us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:57.553682 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:57.553708 (+    26us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:57.554364 (+   656us) tablet.cc:701] Row locks acquired
0709 13:02:57.554417 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:02:57.554575 (+   158us) write_op.cc:270] Start()
0709 13:02:57.554736 (+   161us) write_op.cc:276] Timestamp: P: 1783602177554510 usec, L: 0
0709 13:02:57.554776 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:02:57.610053 (+ 55277us) log.cc:844] Serialized 2069 byte log entry
0709 13:02:58.331017 (+720964us) op_driver.cc:464] REPLICATION: finished
0709 13:02:58.547012 (+215995us) write_op.cc:301] APPLY: starting
0709 13:02:58.547137 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:58.552412 (+  5275us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:58.552568 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:58.617097 (+ 64529us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:58.617438 (+   341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:02:58.617499 (+    61us) write_op.cc:312] APPLY: finished
0709 13:02:58.623492 (+  5993us) log.cc:844] Serialized 57 byte log entry
0709 13:02:58.623703 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:58.623816 (+   113us) write_op.cc:454] Released schema lock
0709 13:02:58.623873 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:58.624244 (+   371us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:58.624273 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":901130,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":205766,"apply.run_cpu_time_us":10727,"apply.run_wall_time_us":87137,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25017,"num_ops":5,"prepare.queue_time_us":308208,"prepare.run_cpu_time_us":26393,"prepare.run_wall_time_us":308386,"raft.queue_time_us":431090,"raft.run_cpu_time_us":33333,"raft.run_wall_time_us":125838,"replication_time_us":776029,"spinlock_wait_cycles":34048,"thread_start_us":729578,"threads_started":4,"wal-append.queue_time_us":247238}]]}
I20260709 13:02:58.997442  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:58.997498  8715 leader_election.cc:304] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:59.000280  8789 leader_election.cc:336] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.000728  8789 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:59.001533 16525 raft_consensus.cc:2749] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:59.001773  8789 leader_election.cc:336] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:58.992760  8638 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 11 to 14, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 14 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.002256  8789 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:59.006562 16569 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:59.007874  8789 leader_election.cc:336] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.008291  8789 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:59.009042 16569 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:58.999821  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.002794 16562 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.018514 16299 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.019174  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.022089 16299 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.024288  8638 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 10 to 17, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 17 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:59.032325  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.032965  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.037317  8789 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.037778  8789 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:02:59.038548 16562 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.037989  8638 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.045596 16562 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.040246  8638 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:59.050603  8695 auto_rebalancer.cc:629] Failed to schedule move for tablet f091122e10604f27acd758d1f8da8081: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.055961  8824 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:02:59.058451  8638 catalog_manager.cc:5697] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 12 to 20, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.062716  8638 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 9 to 15, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 15 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.064711  8638 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 16 to 17, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.066663  8638 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.068313  8638 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 11 to 17, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 17 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.075606  8893 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:02:59.078243  8788 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44b3244bf12f42aba79421575a96c621. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.078578 16569 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:02:59.084607  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:59.098007  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.104627 16569 raft_consensus.cc:2955] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:02:59.106578  8893 raft_consensus.cc:2955] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
W20260709 13:02:59.135569  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.137715  8824 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:02:59.145605 16525 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:59.146287 16525 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.151319  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:02:59.158810  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.166940  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:59.168478  8713 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.169425 16299 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.170234 16525 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:59.191732  8893 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:59.193150 16525 raft_consensus.cc:493] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:59.193746 16525 raft_consensus.cc:515] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.197207  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:59.197662 16525 leader_election.cc:290] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:59.201428  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75401b4ca97a4c1eb75a016ff2124379" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:59.203137  8712 leader_election.cc:304] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.204337 16525 raft_consensus.cc:2749] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:02:59.206825  8643 catalog_manager.cc:5697] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:59.208145  8789 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.208369  8643 catalog_manager.cc:5697] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 23, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 23 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.209025 16330 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:59.209949  8643 catalog_manager.cc:5697] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:59.210608  8788 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 -> Peer 112105225cc140459688be5d31dcb071 (127.7.117.132:38693): Couldn't send request to peer 112105225cc140459688be5d31dcb071. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3e6b55229c842abb4b6a8551191af12. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.211505  8643 catalog_manager.cc:5697] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:59.214236  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.212896  8643 catalog_manager.cc:5697] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 22 to 24, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 24 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.218873  8643 catalog_manager.cc:5697] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:59.219458  8638 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 202) took 27652 ms (client timeout 15000 ms). Trace:
I20260709 13:02:59.220425  8643 catalog_manager.cc:5697] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 20 to 21, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.232862  8643 catalog_manager.cc:5697] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.234551  8643 catalog_manager.cc:5697] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:59.230409  8638 rpcz_store.cc:269] 0709 13:02:31.567452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.567692 (+   240us) service_pool.cc:224] Handling call
0709 13:02:59.219364 (+27651672us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:59.080044 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:59.080604 (+   560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:59.080645 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:02:59.080662 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:59.081600 (+   938us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:59.081738 (+   138us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:59.081764 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:02:59.085325 (+  3561us) tablet.cc:701] Row locks acquired
0709 13:02:59.085347 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:02:59.085505 (+   158us) write_op.cc:270] Start()
0709 13:02:59.085617 (+   112us) write_op.cc:276] Timestamp: P: 1783602179085455 usec, L: 0
0709 13:02:59.085634 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:02:59.086265 (+   631us) log.cc:844] Serialized 3597 byte log entry
0709 13:02:59.099817 (+ 13552us) op_driver.cc:464] REPLICATION: finished
0709 13:02:59.100138 (+   321us) write_op.cc:301] APPLY: starting
0709 13:02:59.100214 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:59.114268 (+ 14054us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:59.114368 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:59.120643 (+  6275us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:59.120847 (+   204us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:02:59.120888 (+    41us) write_op.cc:312] APPLY: finished
0709 13:02:59.121152 (+   264us) log.cc:844] Serialized 89 byte log entry
0709 13:02:59.126259 (+  5107us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.126397 (+   138us) write_op.cc:454] Released schema lock
0709 13:02:59.126449 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:59.126858 (+   409us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.126888 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1774420,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":12375,"apply.run_wall_time_us":26640,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37602,"num_ops":9,"prepare.queue_time_us":2056,"prepare.run_cpu_time_us":3699,"prepare.run_wall_time_us":6514,"replication_time_us":14017,"thread_start_us":19725,"threads_started":1}]]}
I20260709 13:02:59.236111  8643 catalog_manager.cc:5697] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 22 to 23, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 23 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.245591 16496 raft_consensus.cc:493] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.246212 16496 raft_consensus.cc:515] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.248560 16496 leader_election.cc:290] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:59.250245  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:59.254851  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa55bf932444ccebb2b7f6583ead84f" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:59.257946  8713 leader_election.cc:304] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:59.259562 16299 raft_consensus.cc:2749] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:59.238755  8643 catalog_manager.cc:5697] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 17, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 17 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:59.261617  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.260660  8643 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 19 to 21, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 21 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:59.268812  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.273406  8643 catalog_manager.cc:5697] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 20 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.275751  8643 catalog_manager.cc:5697] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.277092  8643 catalog_manager.cc:5697] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:59.278118  7638 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 8774)
I20260709 13:02:59.278430  8643 catalog_manager.cc:5697] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.279816  8643 catalog_manager.cc:5697] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 19, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 19 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.290234  8643 catalog_manager.cc:5697] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:59.291610  8643 catalog_manager.cc:5697] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 9 to 18, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
W20260709 13:02:59.289335  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.292965  8643 catalog_manager.cc:5697] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.294713  8643 catalog_manager.cc:5697] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 21 to 22, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 22 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.296437  8643 catalog_manager.cc:5697] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 14 to 20, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 20 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.298291  8643 catalog_manager.cc:5697] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 10 to 18, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 18 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.299621  8643 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.301359  8643 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.302778  8643 catalog_manager.cc:5697] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 12 to 17, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 17 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.303689  8643 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 7 to 19. New cstate: current_term: 19 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.304786  8643 catalog_manager.cc:5697] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.305073 16569 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:02:59.306030  8643 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 15 to 16, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 16 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.307756  8643 catalog_manager.cc:5697] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.308571  8898 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } } }
I20260709 13:02:59.309398  8643 catalog_manager.cc:5697] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 21, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.311039  8643 catalog_manager.cc:5697] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 19 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 22 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.312107 16299 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:02:59.312783 16299 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.312813  8643 catalog_manager.cc:5697] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 21, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.314288  8643 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 21 to 24, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 24 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.315793  8643 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.317592 16299 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:59.317184  8643 catalog_manager.cc:5697] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 20 to 22, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.319805  8643 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 18, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 18 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:02:59.318403  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:59.321153  8643 catalog_manager.cc:5697] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:59.329802  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:59.332571  8712 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.335958 16299 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.361686  8824 raft_consensus.cc:1275] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:59.363647 16244 consensus_queue.cc:1048] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:02:59.381587  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.400350  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.407366 16299 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:02:59.407953 16299 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:59.410130 16299 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:59.412309  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:59.418141  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011f1e8a4b474eddb6b217393cbdd488" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:59.419812  8715 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:02:59.420825 16299 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:59.443169  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.473575  9414 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 224 similar messages]
W20260709 13:02:59.482286  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.507182 16299 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.510703 16299 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:59.512969 16299 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:02:59.514237  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.514941  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:59.519132  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:59.516685  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:59.520375  8824 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 19.
I20260709 13:02:59.521379  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:59.522701 16496 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:59.523097 16496 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.523439 16496 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.550827 16496 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:59.551723  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.555413 16496 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:59.557934  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:02:59.558228 16525 raft_consensus.cc:493] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.558631  8821 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.558833 16525 raft_consensus.cc:515] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.557513  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:02:59.561661  8893 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.562840  9444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:02:59.561216 16525 leader_election.cc:290] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:02:59.569466 16525 raft_consensus.cc:493] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.569991 16525 raft_consensus.cc:515] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:02:59.571796  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3006e1c3fb5b4c66852cbc955265f1cb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:02:59.572233  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.573098  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:02:59.576628  8893 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:59.578253  8712 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52; no voters: 
I20260709 13:02:59.578379  8715 leader_election.cc:304] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:02:59.579187 16299 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:59.578433  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784f5ebc699a4812aaa4be775e13350a" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:02:59.577100 16525 leader_election.cc:290] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:02:59.582079  8821 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60591217ee24378bb66a86d6d0dda52 in term 20.
I20260709 13:02:59.582286 16525 raft_consensus.cc:2749] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:59.589819 16593 ts_tablet_manager.cc:933] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:02:59.592238  8715 leader_election.cc:304] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:02:59.595577  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.596264 16496 raft_consensus.cc:2749] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:02:59.598483  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.602324 16299 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Becoming Leader. State: Replica: b60591217ee24378bb66a86d6d0dda52, State: Running, Role: LEADER
I20260709 13:02:59.603631 16299 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:02:59.611017  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.634186  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.646087 16593 tablet_copy_client.cc:323] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 13:02:59.648376  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet 44b3244bf12f42aba79421575a96c621 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 13:02:59.649044  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet 44b3244bf12f42aba79421575a96c621 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-44b3244bf12f42aba79421575a96c621
I20260709 13:02:59.659123  8834 tablet_copy_source_session.cc:215] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:59.691011 16593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b3244bf12f42aba79421575a96c621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:59.711930  8646 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 11 to 14, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 14 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:02:59.713762  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.716486  8643 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55196 (request call id 289) took 1460 ms. Trace:
I20260709 13:02:59.726616  8643 rpcz_store.cc:276] 0709 13:02:58.259866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.990576 (+730710us) service_pool.cc:224] Handling call
0709 13:02:59.716403 (+725827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:59.348714 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:59.357659 (+  8945us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:59.357987 (+   328us) write_op.cc:435] Acquired schema lock
0709 13:02:59.358338 (+   351us) tablet.cc:662] Decoding operations
0709 13:02:59.365699 (+  7361us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:59.365740 (+    41us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:59.365759 (+    19us) tablet.cc:685] Acquiring locks for 38 operations
0709 13:02:59.382354 (+ 16595us) tablet.cc:701] Row locks acquired
0709 13:02:59.382375 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:02:59.382524 (+   149us) write_op.cc:270] Start()
0709 13:02:59.382640 (+   116us) write_op.cc:276] Timestamp: P: 1783602179382470 usec, L: 0
0709 13:02:59.382662 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:02:59.383388 (+   726us) log.cc:844] Serialized 15008 byte log entry
0709 13:02:59.387049 (+  3661us) op_driver.cc:464] REPLICATION: finished
0709 13:02:59.388172 (+  1123us) write_op.cc:301] APPLY: starting
0709 13:02:59.388257 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:59.479527 (+ 91270us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:59.479634 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:59.579222 (+ 99588us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:59.579623 (+   401us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=76,key_file_lookups=76,delta_file_lookups=76,mrs_lookups=0
0709 13:02:59.579669 (+    46us) write_op.cc:312] APPLY: finished
0709 13:02:59.580447 (+   778us) log.cc:844] Serialized 324 byte log entry
0709 13:02:59.617440 (+ 36993us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.617749 (+   309us) write_op.cc:454] Released schema lock
0709 13:02:59.617829 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:59.643063 (+ 25234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.643097 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7147,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":853,"apply.run_cpu_time_us":46003,"apply.run_wall_time_us":254851,"cfile_cache_hit":152,"cfile_cache_hit_bytes":158764,"num_ops":38,"prepare.queue_time_us":1172,"prepare.run_cpu_time_us":8590,"prepare.run_wall_time_us":35064,"replication_time_us":4235}]]}
W20260709 13:02:59.747576  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.769846  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.808921  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.814172 16593 tablet_copy_client.cc:806] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:59.814895 16593 tablet_copy_client.cc:670] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 13:02:59.823262 16597 ts_tablet_manager.cc:933] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Initiating tablet copy from peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:02:59.843883  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.845116 16597 tablet_copy_client.cc:323] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: tablet copy: Beginning tablet copy session from remote peer at address 127.7.117.130:44185
I20260709 13:02:59.855347 16593 tablet_copy_client.cc:538] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:02:59.860653  8646 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 273) took 44030 ms (client timeout 15000 ms). Trace:
W20260709 13:02:59.869774  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:59.861133  8646 rpcz_store.cc:269] 0709 13:02:15.856118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.861207 (+  5089us) service_pool.cc:224] Handling call
0709 13:02:16.022054 (+160847us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b5400002210. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33aab71 00007f90f33ab0b2 00007f90fc324a89 00007f90fc2bcda6 00007f90fc28e088 00007f90fc38c37e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc
0709 13:02:59.860567 (+43838513us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:59.723539 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:59.724117 (+   578us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:59.724154 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:02:59.724169 (+    15us) tablet.cc:662] Decoding operations
0709 13:02:59.724521 (+   352us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:59.724582 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:59.724611 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:59.724799 (+   188us) tablet.cc:701] Row locks acquired
0709 13:02:59.724814 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:02:59.724967 (+   153us) write_op.cc:270] Start()
0709 13:02:59.725086 (+   119us) write_op.cc:276] Timestamp: P: 1783602179724920 usec, L: 0
0709 13:02:59.725109 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:59.725737 (+   628us) log.cc:844] Serialized 536 byte log entry
0709 13:02:59.727448 (+  1711us) op_driver.cc:464] REPLICATION: finished
0709 13:02:59.727887 (+   439us) write_op.cc:301] APPLY: starting
0709 13:02:59.727956 (+    69us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:59.728708 (+   752us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:59.728791 (+    83us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:59.729500 (+   709us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:59.729675 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:02:59.729725 (+    50us) write_op.cc:312] APPLY: finished
0709 13:02:59.729892 (+   167us) log.cc:844] Serialized 25 byte log entry
0709 13:02:59.730046 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.730140 (+    94us) write_op.cc:454] Released schema lock
0709 13:02:59.730178 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:59.730517 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.730543 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30432,"reported_tablets":48,"spinlock_wait_cycles":54222080,"child_traces":[["op",{"apply.queue_time_us":240,"apply.run_cpu_time_us":2580,"apply.run_wall_time_us":2591,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":2482,"prepare.run_wall_time_us":2482,"replication_time_us":2199}]]}
I20260709 13:02:59.869632  8834 tablet_copy_service.cc:140] P 29c7934ee528454785432732d01ed0a0: Received BeginTabletCopySession request for tablet d3e6b55229c842abb4b6a8551191af12 from peer 112105225cc140459688be5d31dcb071 ({username='slave'} at 127.0.0.1:36246)
I20260709 13:02:59.887938 16593 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 13:02:59.890981  8834 tablet_copy_service.cc:161] P 29c7934ee528454785432732d01ed0a0: Beginning new tablet copy session on tablet d3e6b55229c842abb4b6a8551191af12 from peer 112105225cc140459688be5d31dcb071 at {username='slave'} at 127.0.0.1:36246: session id = 112105225cc140459688be5d31dcb071-d3e6b55229c842abb4b6a8551191af12
I20260709 13:02:59.916818  8640 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 8 to 12, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 12 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:59.931533  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.939818  8834 tablet_copy_source_session.cc:215] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:59.947849 16597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3e6b55229c842abb4b6a8551191af12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:59.956332  8643 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 14, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:59.967229  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:59.968374  8643 catalog_manager.cc:5697] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 3 to 11, VOTER 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) evicted. New cstate: current_term: 18 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:02:59.995347 16299 raft_consensus.cc:493] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.995935 16299 raft_consensus.cc:515] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:02:59.998185 16299 leader_election.cc:290] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:00.003233  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.005712  9443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac343934349411088cbf7a05932c2a5" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:03:00.016132  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.018162  8713 leader_election.cc:304] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:00.019418 16299 raft_consensus.cc:2749] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:00.024219  8898 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:00.025879 16299 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:59.979470  8643 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index -1 to 13, NON_VOTER 112105225cc140459688be5d31dcb071 (127.7.117.132) added. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.038810  8643 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:00.050681  8643 catalog_manager.cc:5697] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 20 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:03:00.060981  8643 catalog_manager.cc:5697] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 11 to 26. New cstate: current_term: 26 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:00.062374  8643 catalog_manager.cc:5697] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 23 to 25, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 25 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:00.064199  8643 catalog_manager.cc:5697] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 17 to 21, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:00.066020  8643 catalog_manager.cc:5697] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:00.119247  8824 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer b60591217ee24378bb66a86d6d0dda52: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:03:00.129207 16299 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:00.146431  8640 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 303) took 50549 ms (client timeout 15000 ms). Trace:
I20260709 13:03:00.155161  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e with delete_type TABLET_DATA_TOMBSTONED (TS 7fd110c312c14e17a2bbe6d453493cf2 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:56790
I20260709 13:03:00.136763 16594 raft_consensus.cc:493] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.159289 16594 raft_consensus.cc:515] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:03:00.174207  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.176751 16597 tablet_copy_client.cc:806] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:00.177939 16299 raft_consensus.cc:493] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.178470 16299 raft_consensus.cc:515] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:00.180537 16299 leader_election.cc:290] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:00.182161  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.182564  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.189023  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c782391e02e74664be7b5f81a3a9478d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:00.190451 16597 tablet_copy_client.cc:670] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:00.198764  8715 leader_election.cc:304] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.201229 16603 raft_consensus.cc:2749] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:00.204190 16299 raft_consensus.cc:493] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.204666 16299 raft_consensus.cc:515] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:00.205711 16597 tablet_copy_client.cc:538] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:00.209458 16299 leader_election.cc:290] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:00.210534  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:00.210763  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8087b240e9e4f5d9ea874980d0d7efa" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:00.211926 16594 leader_election.cc:290] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:00.218861  8712 leader_election.cc:304] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.219738  8896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90645da839314a85a4bdd72950c292b9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:00.146968  8640 rpcz_store.cc:269] 0709 13:02:09.596862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.601465 (+  4603us) service_pool.cc:224] Handling call
0709 13:02:11.108574 (+1507109us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b5400002210. stack: 00007f90f33b0343 00007f90f33b0240 00007f90f2d3f1d4 00007f90fca7d304 00007f90fca7d299 00007f90f326adb1 00007f90f33ab640 00007f90fc282b76 00007f90fc38cc3e 00007f90fa84d2a4 00007f90fa84d232 00007f90fa84d161 00007f90fa84d0dc 00007f90fa84c392 00007f90f480a48c 00007f90f48098c6
0709 13:03:00.146350 (+49037776us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:59.929624 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:59.930288 (+   664us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:59.930335 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:02:59.930359 (+    24us) tablet.cc:662] Decoding operations
0709 13:02:59.930784 (+   425us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:59.930851 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:59.930877 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:02:59.931068 (+   191us) tablet.cc:701] Row locks acquired
0709 13:02:59.931088 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:02:59.931241 (+   153us) write_op.cc:270] Start()
0709 13:02:59.931358 (+   117us) write_op.cc:276] Timestamp: P: 1783602179931194 usec, L: 0
0709 13:02:59.931381 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:59.932045 (+   664us) log.cc:844] Serialized 536 byte log entry
0709 13:02:59.933760 (+  1715us) op_driver.cc:464] REPLICATION: finished
0709 13:02:59.934052 (+   292us) write_op.cc:301] APPLY: starting
0709 13:02:59.934160 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:59.934947 (+   787us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:59.935037 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:59.935705 (+   668us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:59.935862 (+   157us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:02:59.935911 (+    49us) write_op.cc:312] APPLY: finished
0709 13:02:59.936114 (+   203us) log.cc:844] Serialized 25 byte log entry
0709 13:02:59.936310 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.936388 (+    78us) write_op.cc:454] Released schema lock
0709 13:02:59.936427 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:59.936749 (+   322us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.936783 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":96197,"reported_tablets":49,"spinlock_wait_cycles":6623104,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":2683,"apply.run_wall_time_us":2683,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":5630,"prepare.run_cpu_time_us":2757,"prepare.run_wall_time_us":2756,"replication_time_us":2189}]]}
I20260709 13:03:00.250653  8712 leader_election.cc:304] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.245527 16597 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Bootstrap starting.
I20260709 13:03:00.249647 16594 raft_consensus.cc:2749] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:00.268632 16593 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:00.257839 16603 raft_consensus.cc:2749] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:00.278680  7638 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 8774)
W20260709 13:03:00.330755  8633 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71612.4us
I20260709 13:03:00.343215 16604 tablet_replica.cc:333] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:00.344341 16604 raft_consensus.cc:2243] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:00.345052 16604 raft_consensus.cc:2272] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:00.328770 16593 tablet_bootstrap.cc:492] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 13:03:00.346117 16593 ts_tablet_manager.cc:1403] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.458s	user 0.049s	sys 0.010s
I20260709 13:03:00.351558 16604 ts_tablet_manager.cc:1916] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:03:00.341952  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): ChangeConfig:REMOVE_PEER RPC failed for tablet fe13736604454be3b391b8e946192961: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:00.382764  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.402879 16593 raft_consensus.cc:359] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:03:00.412117 16593 raft_consensus.cc:740] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 13:03:00.417814 16593 consensus_queue.cc:260] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
W20260709 13:03:00.429319  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:00.452780  8642 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 201) took 55430 ms (client timeout 15000 ms). Trace:
W20260709 13:03:00.453632  8642 rpcz_store.cc:269] 0709 13:02:05.022006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.025493 (+  3487us) service_pool.cc:224] Handling call
0709 13:03:00.452551 (+55427058us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153840,"reported_tablets":35,"spinlock_wait_cycles":17792}
I20260709 13:03:00.456730 16593 ts_tablet_manager.cc:1434] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.001s
W20260709 13:03:00.461691  8787 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 112094us
I20260709 13:03:00.477671  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-44b3244bf12f42aba79421575a96c621 received from {username='slave'} at 127.0.0.1:36246
I20260709 13:03:00.478283  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-44b3244bf12f42aba79421575a96c621 on tablet 44b3244bf12f42aba79421575a96c621 with peer 112105225cc140459688be5d31dcb071
I20260709 13:03:00.467334 16299 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:00.497277 16299 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:00.523716  9448 raft_consensus.cc:1217] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.13->[21.14-21.14]   Dedup: 21.14->[]
I20260709 13:03:00.531338  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:03:00.535666  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 45637.3us
I20260709 13:03:00.540863 16299 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:00.581875  8639 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 14 to 22, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 22 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:00.586380  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 101529us
I20260709 13:03:00.614573  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7884996e8b84433f80124e36111f58ac" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:03:00.627252  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 162634us
W20260709 13:03:00.678007  8713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 64812.9us
I20260709 13:03:00.679632  8713 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:00.680450 16603 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:00.705731 16603 raft_consensus.cc:493] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.706343 16603 raft_consensus.cc:515] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:00.708835 16603 leader_election.cc:290] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:00.710593  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:00.716557  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.719326  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a998af2b1141848fa02e188ffcd3c6" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.721236  8715 leader_election.cc:304] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.736802 16299 raft_consensus.cc:2749] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.782672 16604 ts_tablet_manager.cc:1929] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.10
I20260709 13:03:00.783288 16604 log.cc:1199] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2: Deleting WAL directory at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/wals/d4f573b86fda4b3e8fd982da9615dc8e
I20260709 13:03:00.796983  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet d4f573b86fda4b3e8fd982da9615dc8e (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
W20260709 13:03:00.798522  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.800101  8639 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 15 to 16, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 16 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.915292 16597 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:00.916266 16597 tablet_bootstrap.cc:492] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Bootstrap complete.
I20260709 13:03:00.917038 16597 ts_tablet_manager.cc:1403] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Time spent bootstrapping tablet: real 0.672s	user 0.046s	sys 0.024s
W20260709 13:03:00.944406  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.948487 16299 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.949158 16299 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:00.951339 16299 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:00.956370  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.959463  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318340b5bf3b4640972d271299414f94" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:00.961268  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:00.965456 16626 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.966014  8640 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } }
I20260709 13:03:00.966028 16626 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:00.974699  8712 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.979137  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:00.981492 16299 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.976336 16626 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:00.959661 16597 raft_consensus.cc:359] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:03:00.983664  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:00.984228 16597 raft_consensus.cc:740] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 112105225cc140459688be5d31dcb071, State: Initialized, Role: LEARNER
I20260709 13:03:00.985087 16597 consensus_queue.cc:260] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: NON_VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: true } }
I20260709 13:03:00.989861 16597 ts_tablet_manager.cc:1434] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.000s
I20260709 13:03:00.996117  8715 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:00.999809  8834 tablet_copy_service.cc:342] P 29c7934ee528454785432732d01ed0a0: Request end of tablet copy session 112105225cc140459688be5d31dcb071-d3e6b55229c842abb4b6a8551191af12 received from {username='slave'} at 127.0.0.1:36246
I20260709 13:03:01.000373  8834 tablet_copy_service.cc:434] P 29c7934ee528454785432732d01ed0a0: ending tablet copy session 112105225cc140459688be5d31dcb071-d3e6b55229c842abb4b6a8551191af12 on tablet d3e6b55229c842abb4b6a8551191af12 with peer 112105225cc140459688be5d31dcb071
I20260709 13:03:01.000874 16626 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.042868 16626 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.047911 16626 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.049962 16626 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:01.050503  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.051915  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d3d159f1de4558ab5e2c0dcfc6f4d1" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.053601  8715 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.054648 16626 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:01.029843  8638 catalog_manager.cc:5697] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 16 to 19, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 19 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.059662  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:56790
I20260709 13:03:01.061731  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet d4f573b86fda4b3e8fd982da9615dc8e (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 13:03:01.062186  8638 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 17 to 21, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 21 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.064248  8638 catalog_manager.cc:5697] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 22 to 26, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 26 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.065876  8638 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 14 to 19, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 19 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.068017 16626 raft_consensus.cc:493] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:03:01.067665  8638 catalog_manager.cc:5697] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.068746 16626 raft_consensus.cc:515] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:01.059970  8639 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 304) took 30883 ms (client timeout 15000 ms). Trace:
I20260709 13:03:01.084956 16603 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:01.113534 16626 leader_election.cc:290] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.113552 16603 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.115234  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.115864 16603 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:01.101743  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:01.119270  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:03:01.112197  8639 rpcz_store.cc:269] 0709 13:02:30.176504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.176809 (+   305us) service_pool.cc:224] Handling call
0709 13:03:01.059872 (+30883063us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:00.825271 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:00.825857 (+   586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:00.825899 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:03:00.825920 (+    21us) tablet.cc:662] Decoding operations
0709 13:03:00.826371 (+   451us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:00.826462 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:00.826496 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:00.826835 (+   339us) tablet.cc:701] Row locks acquired
0709 13:03:00.826852 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:03:00.827018 (+   166us) write_op.cc:270] Start()
0709 13:03:00.827143 (+   125us) write_op.cc:276] Timestamp: P: 1783602180826971 usec, L: 0
0709 13:03:00.827165 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:00.827856 (+   691us) log.cc:844] Serialized 919 byte log entry
0709 13:03:00.869891 (+ 42035us) op_driver.cc:464] REPLICATION: finished
0709 13:03:00.906862 (+ 36971us) write_op.cc:301] APPLY: starting
0709 13:03:00.906951 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:00.925639 (+ 18688us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:00.927047 (+  1408us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:00.928440 (+  1393us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:00.928624 (+   184us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:03:00.928677 (+    53us) write_op.cc:312] APPLY: finished
0709 13:03:00.928883 (+   206us) log.cc:844] Serialized 33 byte log entry
0709 13:03:00.941919 (+ 13036us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:00.942007 (+    88us) write_op.cc:454] Released schema lock
0709 13:03:00.942059 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:00.953833 (+ 11774us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:00.953867 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":965772,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":36620,"apply.run_cpu_time_us":4396,"apply.run_wall_time_us":46941,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12483,"num_ops":2,"prepare.queue_time_us":8526,"prepare.run_cpu_time_us":2873,"prepare.run_wall_time_us":2873,"replication_time_us":42581,"thread_start_us":22865,"threads_started":2}]]}
I20260709 13:03:01.123752  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011b370821004185a003be210a525c01" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.125550  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3894b893d1341819b9b2b487f3b2b26" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.132107  8712 leader_election.cc:304] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.133173 16626 raft_consensus.cc:2749] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:01.133455 16613 raft_consensus.cc:1064] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:03:01.136462 16613 consensus_queue.cc:237] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.138334  8712 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:03:01.140203  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:01.141225 16626 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:01.143356  8638 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 12 to 21, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to b60591217ee24378bb66a86d6d0dda52 (127.7.117.129). New cstate: current_term: 21 leader_uuid: "b60591217ee24378bb66a86d6d0dda52" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.149219  8638 catalog_manager.cc:5697] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 17 to 20, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.165040  8638 catalog_manager.cc:5697] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.161496 16603 raft_consensus.cc:493] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:03:01.170951 16603 raft_consensus.cc:515] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.175902  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.176376 16603 leader_election.cc:290] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.177676  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a0644aef66444b58e8633da6993535c" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.178342  8898 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:01.182368  8712 leader_election.cc:304] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.185878 16330 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:01.202406 16603 raft_consensus.cc:2749] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:01.211006  9448 raft_consensus.cc:1275] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:03:01.213096 16611 consensus_queue.cc:1048] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:03:01.214190  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:01.247587  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:01.247656 16626 raft_consensus.cc:493] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:03:01.248309 16626 raft_consensus.cc:515] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.250504 16626 leader_election.cc:290] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.251507  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.254262  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "029b57162d72422bba468c7d86b67fca" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.261937  8715 leader_election.cc:304] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:01.262754 16626 raft_consensus.cc:2749] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:01.279445  7638 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 8774)
I20260709 13:03:01.293695 16626 raft_consensus.cc:493] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:03:01.294363 16626 raft_consensus.cc:515] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.297876  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.301206  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc45d3a2731b49daacb16548dc742815" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.302614  8713 leader_election.cc:304] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.303431 16299 raft_consensus.cc:2749] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.304234 16299 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:01.304693 16299 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.306804 16299 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:01.308605  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.311619  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91581ed67848b791865dfe11e4fc78" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.310818 16626 leader_election.cc:290] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.315270  8715 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.318761 16626 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:01.321173  8641 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 19 to 21, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.329376 16299 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:01.329958 16611 raft_consensus.cc:2955] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.320097  8638 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 275) took 2947 ms. Trace:
I20260709 13:03:01.333719  9448 raft_consensus.cc:2955] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.339479  8641 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 reported cstate change: term changed from 14 to 15, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 15 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.333660  8638 rpcz_store.cc:276] 0709 13:02:58.370516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.255075 (+884559us) service_pool.cc:224] Handling call
0709 13:03:01.318114 (+2063039us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:01.196391 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:01.197003 (+   612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:01.197051 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:03:01.197068 (+    17us) tablet.cc:662] Decoding operations
0709 13:03:01.197965 (+   897us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:01.198048 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:01.198080 (+    32us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:03:01.198796 (+   716us) tablet.cc:701] Row locks acquired
0709 13:03:01.198817 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:03:01.198959 (+   142us) write_op.cc:270] Start()
0709 13:03:01.199087 (+   128us) write_op.cc:276] Timestamp: P: 1783602181198912 usec, L: 0
0709 13:03:01.199113 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:03:01.200235 (+  1122us) log.cc:844] Serialized 3218 byte log entry
0709 13:03:01.202339 (+  2104us) op_driver.cc:464] REPLICATION: finished
0709 13:03:01.202661 (+   322us) write_op.cc:301] APPLY: starting
0709 13:03:01.202741 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:01.206869 (+  4128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:01.206970 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:01.226683 (+ 19713us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:01.226877 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:03:01.226919 (+    42us) write_op.cc:312] APPLY: finished
0709 13:03:01.234083 (+  7164us) log.cc:844] Serialized 81 byte log entry
0709 13:03:01.234271 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.234407 (+   136us) write_op.cc:454] Released schema lock
0709 13:03:01.234466 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:01.235576 (+  1110us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.235611 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2062,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":124,"apply.run_cpu_time_us":10717,"apply.run_wall_time_us":32874,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33424,"num_ops":8,"prepare.queue_time_us":6693,"prepare.run_cpu_time_us":3866,"prepare.run_wall_time_us":5909,"replication_time_us":3105}]]}
I20260709 13:03:01.349396  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:01.352197 16603 raft_consensus.cc:493] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.352731 16603 raft_consensus.cc:515] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.356627  7638 tablet_replica.cc:333] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.357713  7638 raft_consensus.cc:2243] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.358494  7638 raft_consensus.cc:2272] T c6ea7c9c3095497eb93f5d5d412ad667 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.361554  9442 raft_consensus.cc:1217] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:03:01.366767  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.367357  8893 raft_consensus.cc:1803] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: voting while tombstoned based on last-logged opid 18.10
I20260709 13:03:01.367908  8893 raft_consensus.cc:2410] T d4f573b86fda4b3e8fd982da9615dc8e P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b60591217ee24378bb66a86d6d0dda52 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:03:01.369536  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): ChangeConfig:REMOVE_PEER RPC failed for tablet 31c225a50048451db8964738c5db7540: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:01.329964 16299 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.371882  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.385097  8898 raft_consensus.cc:2955] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:01.386376 16603 leader_election.cc:290] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.389446  7638 tablet_replica.cc:333] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.387943  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f573b86fda4b3e8fd982da9615dc8e" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.389176 16299 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:01.390465  7638 raft_consensus.cc:2243] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.394434  7638 raft_consensus.cc:2272] T 31c225a50048451db8964738c5db7540 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.397413  7638 tablet_replica.cc:333] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.398190  7638 raft_consensus.cc:2243] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.414278  7638 raft_consensus.cc:2272] T a07b06e72d474091a0bcf7de181b509f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.415226  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.415688  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b2ea39860b462cb7b87b739253a0c9" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.416152  8713 leader_election.cc:304] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.426707 16299 raft_consensus.cc:2749] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:01.435086  7638 tablet_replica.cc:333] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.436125  7638 raft_consensus.cc:2243] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.436844  7638 raft_consensus.cc:2272] T 7884996e8b84433f80124e36111f58ac P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.437626  8715 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.443719  7638 tablet_replica.cc:333] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.444654  7638 raft_consensus.cc:2243] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.445422  7638 raft_consensus.cc:2272] T 3006e1c3fb5b4c66852cbc955265f1cb P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.448227  7638 tablet_replica.cc:333] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.443629 16299 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:01.449891  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:01.453554  7638 raft_consensus.cc:2243] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.454357  7638 raft_consensus.cc:2272] T dc6dcf2342014ed6b65dd5af29d18d3b P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.459416  7638 tablet_replica.cc:333] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.460242  7638 raft_consensus.cc:2243] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.460942  7638 raft_consensus.cc:2272] T 265e5f9784a6426482f5948fbfcc995c P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.463758  7638 tablet_replica.cc:333] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.464545  7638 raft_consensus.cc:2243] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.465255  7638 raft_consensus.cc:2272] T 784f5ebc699a4812aaa4be775e13350a P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.468124  7638 tablet_replica.cc:333] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.468988  7638 raft_consensus.cc:2243] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.469897  7638 pending_rounds.cc:70] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 2 pending ops.
I20260709 13:03:01.470120  7638 pending_rounds.cc:77] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305634500605546496 op_type: NO_OP noop_request { }
I20260709 13:03:01.470530  7638 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.470816  7638 pending_rounds.cc:77] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 20 index: 10 } timestamp: 7305634482201018368 op_type: NO_OP noop_request { }
I20260709 13:03:01.471139  7638 raft_consensus.cc:2889] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.471446  7638 raft_consensus.cc:2272] T b3894b893d1341819b9b2b487f3b2b26 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.474195  7638 tablet_replica.cc:333] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.474932  7638 raft_consensus.cc:2243] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:03:01.485898  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:01.486766  7638 raft_consensus.cc:2272] T e29b7d4482f943ce897e5a8d5bbb48c4 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.489636  7638 tablet_replica.cc:333] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.490546  7638 raft_consensus.cc:2243] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.490175  8645 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 21 to 22, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.491137  7638 raft_consensus.cc:2272] T f1a7b40e33c24527acba97c95cee9272 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.492862 16299 raft_consensus.cc:493] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 112105225cc140459688be5d31dcb071)
I20260709 13:03:01.493422 16299 raft_consensus.cc:515] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.493975  7638 tablet_replica.cc:333] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.495147  7638 raft_consensus.cc:2243] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.495585 16299 leader_election.cc:290] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.495743  7638 raft_consensus.cc:2272] T 0eb392189d54431c863a3b787fb95779 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.498286  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.498843  7638 tablet_replica.cc:333] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.499698  7638 raft_consensus.cc:2243] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.500242  9442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f192c6b771043a893d6fd420c9b9e7d" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.500432  7638 raft_consensus.cc:2272] T d4f573b86fda4b3e8fd982da9615dc8e P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.501686  8713 leader_election.cc:304] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.503110 16299 raft_consensus.cc:2749] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:01.505223  8641 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55184 (request call id 274) took 22357 ms (client timeout 15000 ms). Trace:
I20260709 13:03:01.506811  7638 tablet_replica.cc:333] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.511729  7638 raft_consensus.cc:2243] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.512406  7638 pending_rounds.cc:70] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 1 pending ops.
I20260709 13:03:01.512703  7638 pending_rounds.cc:77] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305634519014551552 op_type: NO_OP noop_request { }
I20260709 13:03:01.512106  8645 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 15 to 16, leader changed from 112105225cc140459688be5d31dcb071 (127.7.117.132) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 16 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.513049  7638 raft_consensus.cc:2889] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.513513  7638 raft_consensus.cc:2272] T 4ac343934349411088cbf7a05932c2a5 P b60591217ee24378bb66a86d6d0dda52 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:01.505679  8641 rpcz_store.cc:269] 0709 13:02:39.153112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.153407 (+   295us) service_pool.cc:224] Handling call
0709 13:03:01.505146 (+22351739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:01.361462 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:01.361967 (+   505us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:01.362016 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:03:01.362029 (+    13us) tablet.cc:662] Decoding operations
0709 13:03:01.362416 (+   387us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:01.362463 (+    47us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:01.362494 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:01.362771 (+   277us) tablet.cc:701] Row locks acquired
0709 13:03:01.362783 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:03:01.362922 (+   139us) write_op.cc:270] Start()
0709 13:03:01.363038 (+   116us) write_op.cc:276] Timestamp: P: 1783602181362879 usec, L: 0
0709 13:03:01.363053 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:03:01.363605 (+   552us) log.cc:844] Serialized 915 byte log entry
0709 13:03:01.365471 (+  1866us) op_driver.cc:464] REPLICATION: finished
0709 13:03:01.365788 (+   317us) write_op.cc:301] APPLY: starting
0709 13:03:01.365871 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:01.367197 (+  1326us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:01.367288 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:01.368551 (+  1263us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:01.368726 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:03:01.368778 (+    52us) write_op.cc:312] APPLY: finished
0709 13:03:01.368971 (+   193us) log.cc:844] Serialized 33 byte log entry
0709 13:03:01.369144 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.369232 (+    88us) write_op.cc:454] Released schema lock
0709 13:03:01.369274 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:01.374426 (+  5152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.374457 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":805583,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":119,"apply.run_cpu_time_us":3815,"apply.run_wall_time_us":8590,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8356,"num_ops":2,"prepare.queue_time_us":526,"prepare.run_cpu_time_us":2380,"prepare.run_wall_time_us":2376,"replication_time_us":2308}]]}
I20260709 13:03:01.516464  7638 tablet_replica.cc:333] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.517433  7638 raft_consensus.cc:2243] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.518100  7638 raft_consensus.cc:2272] T 83a998af2b1141848fa02e188ffcd3c6 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.531651  7638 tablet_replica.cc:333] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.532589  7638 raft_consensus.cc:2243] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.540311  8640 catalog_manager.cc:5697] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 14 to 15, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.533222  7638 raft_consensus.cc:2272] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.545538  7638 tablet_replica.cc:333] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.545928 16603 raft_consensus.cc:493] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:01.546406  7638 raft_consensus.cc:2243] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.546489 16603 raft_consensus.cc:515] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.547616  7638 raft_consensus.cc:2272] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.551308  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.553748  7638 tablet_replica.cc:333] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.554756  7638 raft_consensus.cc:2243] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.555680  7638 raft_consensus.cc:2272] T 3f192c6b771043a893d6fd420c9b9e7d P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.560142 16603 leader_election.cc:290] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:01.564417  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3dbb5cac024c52a371cbe4f0f1e952" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.567198  8712 leader_election.cc:304] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.569104 16603 raft_consensus.cc:2749] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:01.569483  7638 tablet_replica.cc:333] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.572129  7638 raft_consensus.cc:2243] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.572842  7638 raft_consensus.cc:2272] T 029b57162d72422bba468c7d86b67fca P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.575809  7638 tablet_replica.cc:333] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.576721  7638 raft_consensus.cc:2243] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.577371  7638 raft_consensus.cc:2272] T 011f1e8a4b474eddb6b217393cbdd488 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.580068  7638 tablet_replica.cc:333] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.580895  7638 raft_consensus.cc:2243] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.591801  7638 raft_consensus.cc:2272] T 318340b5bf3b4640972d271299414f94 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.594835  7638 tablet_replica.cc:333] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.595691  7638 raft_consensus.cc:2243] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.596360  7638 raft_consensus.cc:2272] T 42d038e54cdc4dfaa0cfec37a43480ee P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.610883  7638 tablet_replica.cc:333] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.611791  7638 raft_consensus.cc:2243] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.612114 16603 raft_consensus.cc:493] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.612650 16603 raft_consensus.cc:515] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.615034 16603 leader_election.cc:290] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.612393  7638 pending_rounds.cc:70] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Trying to abort 1 pending ops.
I20260709 13:03:01.616377  7638 pending_rounds.cc:77] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305634498919882752 op_type: NO_OP noop_request { }
I20260709 13:03:01.616561  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.616868  7638 raft_consensus.cc:2889] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.617267  7638 raft_consensus.cc:2272] T de733f99bf86451da507373e7f4b78aa P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:01.721238  8715 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 103148us
I20260709 13:03:01.731949 16603 raft_consensus.cc:493] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.732564  7638 tablet_replica.cc:333] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.732530 16603 raft_consensus.cc:515] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:01.733577  7638 raft_consensus.cc:2243] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.734256  7638 pending_rounds.cc:70] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 1 pending ops.
I20260709 13:03:01.734468  7638 pending_rounds.cc:77] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305634500475375616 op_type: NO_OP noop_request { }
I20260709 13:03:01.734800  7638 raft_consensus.cc:2889] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.735139  7638 raft_consensus.cc:2272] T eac82cfba1d244139d9b06d8a6c88635 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.735203 16603 leader_election.cc:290] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:01.744768  8815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:01.754341  7638 tablet_replica.cc:333] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.783789  7638 raft_consensus.cc:2243] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.788295  7638 raft_consensus.cc:2272] T c5bf6ec0e6a745d0967928a3361eb8d8 P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.791803  7638 tablet_replica.cc:333] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.792925  7638 raft_consensus.cc:2243] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.800514  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53864dd6ef544420915d557a63d5cd91" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.809257  7638 raft_consensus.cc:2272] T 53b2ea39860b462cb7b87b739253a0c9 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.812399  8713 leader_election.cc:304] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:01.813411 16299 raft_consensus.cc:2749] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:01.817318  8789 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:01.822324 16603 raft_consensus.cc:493] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.823021 16603 raft_consensus.cc:515] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:01.828414  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:01.833554  7638 tablet_replica.cc:333] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.834570  7638 raft_consensus.cc:2243] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.836791  7638 raft_consensus.cc:2272] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.839928  7638 tablet_replica.cc:333] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.840708  7638 raft_consensus.cc:2243] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.841753  7638 raft_consensus.cc:2272] T 9402cffd3f284d28bf964bd56144a185 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.844465  7638 tablet_replica.cc:333] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.845257  7638 raft_consensus.cc:2243] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.845948  7638 raft_consensus.cc:2272] T 75401b4ca97a4c1eb75a016ff2124379 P b60591217ee24378bb66a86d6d0dda52 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.849601  8880 tablet_service.cc:1558] Processing DeleteTablet for tablet d4f573b86fda4b3e8fd982da9615dc8e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:56790
I20260709 13:03:01.852557  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:01.854684  7638 tablet_replica.cc:333] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.855664  7638 raft_consensus.cc:2243] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.856251  7638 raft_consensus.cc:2272] T 53864dd6ef544420915d557a63d5cd91 P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.860055  7638 tablet_replica.cc:333] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.861076  7638 raft_consensus.cc:2243] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.861703  7638 raft_consensus.cc:2272] T f091122e10604f27acd758d1f8da8081 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:01.849542  8645 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55202 (request call id 305) took 13569 ms (client timeout 15000 ms). Trace:
I20260709 13:03:01.865219  7638 tablet_replica.cc:333] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.873535 16613 raft_consensus.cc:1064] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: attempting to promote NON_VOTER 112105225cc140459688be5d31dcb071 to VOTER
I20260709 13:03:01.875813  7638 raft_consensus.cc:2243] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.876590  7638 raft_consensus.cc:2272] T 1a0644aef66444b58e8633da6993535c P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.876027 16613 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:01.879678  7638 tablet_replica.cc:333] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.880817  7638 raft_consensus.cc:2243] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.881330  7638 pending_rounds.cc:70] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 2 pending ops.
I20260709 13:03:01.881402 16603 leader_election.cc:290] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:01.881551  7638 pending_rounds.cc:77] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305634519874199552 op_type: NO_OP noop_request { }
I20260709 13:03:01.882479  7638 raft_consensus.cc:2889] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.882841  7638 pending_rounds.cc:77] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 18 index: 10 } timestamp: 7305634496217456640 op_type: NO_OP noop_request { }
I20260709 13:03:01.883265  7638 raft_consensus.cc:2889] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.883601  7638 raft_consensus.cc:2272] T 90645da839314a85a4bdd72950c292b9 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.886678  7638 tablet_replica.cc:333] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.886670  9448 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 LEARNER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:01.887470  8631 catalog_manager.cc:5028] TS 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): tablet d4f573b86fda4b3e8fd982da9615dc8e (table test-workload [id=dec811e65e0b4bb6a470ee6b21001496]) successfully deleted
I20260709 13:03:01.888227  7638 raft_consensus.cc:2243] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.889025  7638 raft_consensus.cc:2272] T d8087b240e9e4f5d9ea874980d0d7efa P b60591217ee24378bb66a86d6d0dda52 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.889483 16642 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:01.891969  7638 tablet_replica.cc:333] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.892978  7638 raft_consensus.cc:2243] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.893633  7638 raft_consensus.cc:2272] T c24c6743859d43109fabeaf7ee8c8149 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.896299  7638 tablet_replica.cc:333] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
W20260709 13:03:01.864851  8645 rpcz_store.cc:269] 0709 13:02:48.279805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.016125 (+2736320us) service_pool.cc:224] Handling call
0709 13:03:01.849455 (+10833330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:01.515875 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:01.516388 (+   513us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:01.516446 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:03:01.516464 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:01.516891 (+   427us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:01.516940 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:01.516965 (+    25us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:01.517300 (+   335us) tablet.cc:701] Row locks acquired
0709 13:03:01.517320 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:03:01.517460 (+   140us) write_op.cc:270] Start()
0709 13:03:01.517571 (+   111us) write_op.cc:276] Timestamp: P: 1783602181517415 usec, L: 0
0709 13:03:01.517593 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:01.518239 (+   646us) log.cc:844] Serialized 915 byte log entry
0709 13:03:01.521307 (+  3068us) op_driver.cc:464] REPLICATION: finished
0709 13:03:01.521634 (+   327us) write_op.cc:301] APPLY: starting
0709 13:03:01.521715 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:01.522991 (+  1276us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:01.523077 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:01.524349 (+  1272us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:01.524512 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:03:01.524555 (+    43us) write_op.cc:312] APPLY: finished
0709 13:03:01.524743 (+   188us) log.cc:844] Serialized 33 byte log entry
0709 13:03:01.527533 (+  2790us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.527620 (+    87us) write_op.cc:454] Released schema lock
0709 13:03:01.527668 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:01.531775 (+  4107us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:01.531808 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35801,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":139,"apply.run_cpu_time_us":3746,"apply.run_wall_time_us":10112,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8356,"num_ops":2,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":2638,"prepare.run_wall_time_us":2637,"replication_time_us":3586}]]}
I20260709 13:03:01.905714  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b476fe65810048fdb66dbeff393c8ccb" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.908206  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd126387f2fb45f98d7ca0807566c688" candidate_uuid: "b60591217ee24378bb66a86d6d0dda52" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:01.911479  8644 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet d4f573b86fda4b3e8fd982da9615dc8e with cas_config_opid_index 3: aborting the task: latest config opid_index 11; task opid_index 3
W20260709 13:03:01.923648  8644 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 203) took 12240 ms (client timeout 15000 ms). Trace:
W20260709 13:03:01.924034  8644 rpcz_store.cc:269] 0709 13:02:49.683236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.678997 (+6995761us) service_pool.cc:224] Handling call
0709 13:03:01.923556 (+5244559us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245849,"reported_tablets":35}
I20260709 13:03:01.925092  7638 raft_consensus.cc:2243] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.925495  8893 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:01.925812  8712 leader_election.cc:304] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:03:01.926529  8712 raft_consensus.cc:2686] T b476fe65810048fdb66dbeff393c8ccb P b60591217ee24378bb66a86d6d0dda52: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:03:01.931427  8789 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:01.931792  7638 raft_consensus.cc:2272] T 6fa55bf932444ccebb2b7f6583ead84f P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.936865  7638 tablet_replica.cc:333] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.941776  7638 raft_consensus.cc:2243] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.942829  7638 raft_consensus.cc:2272] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.945566  7638 tablet_replica.cc:333] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.946285  7638 raft_consensus.cc:2243] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.947108  7638 raft_consensus.cc:2272] T 108cbfcea5a2457d89f3cf70cf52b3ad P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.956753  8712 leader_election.cc:304] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b60591217ee24378bb66a86d6d0dda52; no voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:01.960304 16603 raft_consensus.cc:2749] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:01.968214  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:01.975207  8647 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52550 (request call id 200) took 78016 ms (client timeout 15000 ms). Trace:
W20260709 13:03:01.981829  8647 rpcz_store.cc:269] 0709 13:01:43.967442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.969903 (+  2461us) service_pool.cc:224] Handling call
0709 13:03:01.975020 (+78005117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":965099,"reported_tablets":35,"spinlock_wait_cycles":1916672}
I20260709 13:03:01.976259 16613 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:01.981482  7638 tablet_replica.cc:333] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.983464  7638 raft_consensus.cc:2243] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.984068  7638 raft_consensus.cc:2272] T c782391e02e74664be7b5f81a3a9478d P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.986903  7638 tablet_replica.cc:333] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:01.987834  7638 raft_consensus.cc:2243] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.988680  7638 pending_rounds.cc:70] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Trying to abort 1 pending ops.
I20260709 13:03:01.988941  7638 pending_rounds.cc:77] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305634500608430080 op_type: NO_OP noop_request { }
I20260709 13:03:01.989269  7638 raft_consensus.cc:2889] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:01.989635  7638 raft_consensus.cc:2272] T 4c91581ed67848b791865dfe11e4fc78 P b60591217ee24378bb66a86d6d0dda52 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:02.005807  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:02.009917  8644 catalog_manager.cc:5697] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 16 to 17, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 17 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.008482  7638 tablet_replica.cc:333] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.014123  7638 raft_consensus.cc:2243] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:02.015316  7638 raft_consensus.cc:2272] T 1f2e6f6b24e84190b722a8e69bbe97fa P b60591217ee24378bb66a86d6d0dda52 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:02.024693  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.026906  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.027890  7638 tablet_replica.cc:333] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.028954  7638 raft_consensus.cc:2243] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:03:02.030287  7638 raft_consensus.cc:2272] T fe13736604454be3b391b8e946192961 P b60591217ee24378bb66a86d6d0dda52 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.033339  7638 tablet_replica.cc:333] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.021396  8644 catalog_manager.cc:5697] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 22 to 24, leader changed from 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 24 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:02.026947  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.035825  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.038151  7638 raft_consensus.cc:2243] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:02.052161  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.054755 16642 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:02.056742  8893 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:02.073333  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.074898  7638 raft_consensus.cc:2272] T cb3dbb5cac024c52a371cbe4f0f1e952 P b60591217ee24378bb66a86d6d0dda52 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:02.079293  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.096410  9448 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:02.098290  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.103816  7638 tablet_replica.cc:333] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.104898  7638 raft_consensus.cc:2243] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:02.105687  7638 raft_consensus.cc:2272] T 011b370821004185a003be210a525c01 P b60591217ee24378bb66a86d6d0dda52 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.109127  7638 tablet_replica.cc:333] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.109910  7638 raft_consensus.cc:2243] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:02.110487  7638 raft_consensus.cc:2272] T d3e6b55229c842abb4b6a8551191af12 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.113205  7638 tablet_replica.cc:333] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.116974  7638 raft_consensus.cc:2243] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:02.121780  7638 raft_consensus.cc:2272] T 44b3244bf12f42aba79421575a96c621 P b60591217ee24378bb66a86d6d0dda52 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.124497  7638 tablet_replica.cc:333] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.125177  7638 raft_consensus.cc:2243] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:02.129842  7638 raft_consensus.cc:2272] T fd126387f2fb45f98d7ca0807566c688 P b60591217ee24378bb66a86d6d0dda52 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.132480  7638 tablet_replica.cc:333] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52: stopping tablet replica
I20260709 13:03:02.133140  7638 raft_consensus.cc:2243] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:02.139173  7638 raft_consensus.cc:2272] T fc45d3a2731b49daacb16548dc742815 P b60591217ee24378bb66a86d6d0dda52 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:02.173807  8647 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 reported cstate change: config changed from index 13 to 14, 112105225cc140459688be5d31dcb071 (127.7.117.132) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.203327  8822 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.211408  8893 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:02.215813  9448 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:02.217510 16330 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:02.224888 16330 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:02.227519 16611 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:02.229502  8893 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:02.232290  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.240097  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.239132  9448 raft_consensus.cc:2955] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
I20260709 13:03:02.271574  8634 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d3e6b55229c842abb4b6a8551191af12 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:03:02.287947  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.295553  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.311715  8647 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 reported cstate change: config changed from index 14 to 15, VOTER b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) evicted. New cstate: current_term: 20 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:02.321609  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.384138  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:03:02.399444  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.406023  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.424989  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.426599  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.465332 16616 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.465896 16616 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:02.468116 16616 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:02.470304  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:02.472436  8789 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.476773  8787 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.477955 16616 raft_consensus.cc:2749] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:02.551651  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.558605  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.597242 16640 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.597843 16640 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.600041 16640 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:02.602224  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:03:02.605412  8863 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.605841  8863 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.606940 16643 raft_consensus.cc:2749] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:02.630195  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.668287  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.712548  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.734138  9414 connection.cc:769] client connection to 127.7.117.129:39991 send error: Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 13:03:02.746834 16564 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.747639 16616 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.747483 16564 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.748237 16616 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.752282 16564 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:02.753392  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:02.754810  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:02.759611  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:02.760336  8822 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 24.
I20260709 13:03:02.770648 16616 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:02.772260  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:02.772917  9448 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 24.
I20260709 13:03:02.786441 16616 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.802004 16616 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:02.805517  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:02.806528  8789 connection.cc:769] client connection to 127.7.117.129:39991 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:03:02.809712  8789 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:03:02.817503  8789 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.819563 16330 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:02.822624 16616 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:02.867758  8789 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:02.868330  8789 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.869822 16643 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.870368 16643 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:02.871390 16641 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.873737 16641 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:02.874184 16643 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:02.875802 16641 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:02.879160  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
W20260709 13:03:02.882623  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:02.883188  8863 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:02.884186  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:02.884687  9414 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:02.885076  9414 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.886173  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:02.886984  8821 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:03:02.889242 16564 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.889787 16564 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.891853 16564 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:02.892565 16564 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:02.877488 16330 raft_consensus.cc:2749] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:02.896082  8822 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:03:02.900228  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.901415 16641 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:02.901789 16641 raft_consensus.cc:493] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.902149 16641 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:02.905598  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:02.906587 16643 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:02.907004 16643 raft_consensus.cc:493] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.907441 16643 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:02.920801 16640 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.921433 16640 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.926805  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:02.927654  8821 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:03:02.929078 16640 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:02.938774  8863 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:02.958904  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:02.962198 16636 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.976864 16636 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.979668 16636 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:02.960975 16643 raft_consensus.cc:515] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:02.982995 16616 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:02.983659 16616 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:02.988878 16616 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:02.995127 16641 raft_consensus.cc:515] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.012656 16643 leader_election.cc:290] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:03.023409 16641 leader_election.cc:290] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:02.999202  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 107698us
I20260709 13:03:03.163398  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:03.163997  9445 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 24.
I20260709 13:03:03.195003 16657 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.195631 16657 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.198650 16657 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:03.214532  8863 leader_election.cc:336] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.223491  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:03.228210  8893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:03.228892  8893 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
I20260709 13:03:03.229827  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:03.236222  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:03:03.237068  8863 leader_election.cc:336] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.237793 16659 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:03:03.238018  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.238283 16659 raft_consensus.cc:493] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.238729 16659 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:03.247213  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 362545us
I20260709 13:03:03.259181 16663 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.259992 16663 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.261407 16659 raft_consensus.cc:515] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.262912  8789 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.263031 16663 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:03.263418 16659 leader_election.cc:290] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:03.275942  8862 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.276814 16640 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:03.277195 16640 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.277563 16640 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:03.278069  8788 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 62428.8us
I20260709 13:03:03.282552  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29b7d4482f943ce897e5a8d5bbb48c4" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:03:03.283145  8822 raft_consensus.cc:3060] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:03.296116  8889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a07b06e72d474091a0bcf7de181b509f" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:03:03.296725  8889 raft_consensus.cc:3060] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.294651  8822 raft_consensus.cc:2468] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
I20260709 13:03:03.300009  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9662b1b1994a4a29b1ec6da7c3cd5f39" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:03:03.300666  8821 raft_consensus.cc:3060] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:03.303910  8889 raft_consensus.cc:2468] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:03:03.307849  8821 raft_consensus.cc:2468] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:03:03.308388  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:03.312281 16661 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.312824 16661 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.313462 16564 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.313930 16564 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.314908 16661 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:03.315886 16564 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:03.317178  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:03.321657  8788 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 112105225cc140459688be5d31dcb071, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.322495 16665 raft_consensus.cc:2749] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:03.323086  8864 leader_election.cc:304] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.324033 16664 raft_consensus.cc:2804] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:03.324795 16664 raft_consensus.cc:697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:03:03.325696 16664 consensus_queue.cc:237] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.327500  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:03.328347  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:03.332212  9414 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:03.332978  9414 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.337559  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:03.338387  8821 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 22.
I20260709 13:03:03.342929  9412 leader_election.cc:304] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:03:03.348140 16564 raft_consensus.cc:2804] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:03.288630 16640 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.350436 16564 raft_consensus.cc:697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:03:03.351433 16564 consensus_queue.cc:237] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.355151  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
W20260709 13:03:03.357052  8863 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.357497 16660 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:03.357955 16660 raft_consensus.cc:493] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.358333 16660 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:03.361202  8864 leader_election.cc:304] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.361979 16643 raft_consensus.cc:2804] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:03.362457 16643 raft_consensus.cc:697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:03:03.363202 16643 consensus_queue.cc:237] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.366549  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.367395 16664 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:03.373274  8789 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.373811  8789 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29c7934ee528454785432732d01ed0a0; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.374866 16640 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:03.375865 16616 raft_consensus.cc:2749] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.378216  9414 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:03.385874  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 52926.4us
I20260709 13:03:03.386584  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:03:03.389478 16661 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:03.389634  8822 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:03.391712  8640 catalog_manager.cc:5697] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 19 to 20, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 20 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:03.396471  8822 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 25.
W20260709 13:03:03.404633  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 61798.5us
I20260709 13:03:03.405071 16643 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.405644 16643 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.408527 16643 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:03.402119  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:03.410914 16660 raft_consensus.cc:515] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.411844  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:03.412184  8863 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.414891 16643 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.415469 16643 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.417593 16640 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.418102 16640 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.420614 16643 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:03.422578  8863 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.423337  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:03.425086  8821 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:03:03.428757  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:03.429416  9445 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
W20260709 13:03:03.429919  8863 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.429430  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31c225a50048451db8964738c5db7540" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:03:03.430677  9448 raft_consensus.cc:3060] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:03.420636 16640 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:03.438441  9448 raft_consensus.cc:2468] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 25.
I20260709 13:03:03.439977  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:03.417212  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:03.441640 16660 leader_election.cc:290] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:03.449745 16643 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:03.450168 16643 raft_consensus.cc:493] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.450553 16643 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.452251  8822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f2e6f6b24e84190b722a8e69bbe97fa" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:03:03.452870  8822 raft_consensus.cc:3060] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:03.458586 16643 raft_consensus.cc:515] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:03:03.465392  8863 leader_election.cc:336] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.478020 16564 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.478652 16564 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.481066  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:03.481779  9448 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 19.
I20260709 13:03:03.483702 16643 leader_election.cc:290] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:03.485097  8647 catalog_manager.cc:5697] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 21 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 22 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:03.488859  9414 leader_election.cc:336] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.478653  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.486479  8824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "108cbfcea5a2457d89f3cf70cf52b3ad" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0"
I20260709 13:03:03.489822  8824 raft_consensus.cc:3060] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.490190 16657 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:03.490599 16657 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.490975 16657 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.486855  8821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "29c7934ee528454785432732d01ed0a0" is_pre_election: true
I20260709 13:03:03.488193  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:03.493752  8898 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 16.
I20260709 13:03:03.498560 16657 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.505673  8863 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.506338  8824 raft_consensus.cc:2468] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:03:03.506807 16564 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:03.510815  8822 raft_consensus.cc:2468] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 23.
I20260709 13:03:03.510776  8864 leader_election.cc:304] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.513909 16640 raft_consensus.cc:2804] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:03.514460 16640 raft_consensus.cc:697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:03:03.515372 16640 consensus_queue.cc:237] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:03.511811 16657 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:03.521461  9415 leader_election.cc:304] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0; no voters: 
I20260709 13:03:03.522456 16564 raft_consensus.cc:2804] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:03.523005 16564 raft_consensus.cc:697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:03:03.524037  8862 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.524267 16564 consensus_queue.cc:237] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.526013  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:03.527066  9414 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.527490  9414 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.528957  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:03.529765  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.530500 16660 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.525242 16643 raft_consensus.cc:2804] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:03.541944 16643 raft_consensus.cc:697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:03:03.544332  9414 leader_election.cc:336] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.539999 16657 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:03.547724 16657 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.548163 16657 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:03.550361 16616 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.550900 16616 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.545992 16643 consensus_queue.cc:237] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.559068  9414 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.560155  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.561161 16564 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:03.563331  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bff3e7dddcd4dbebe38db1fa4eaae71" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:03:03.562805  8647 catalog_manager.cc:5697] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 22 to 23, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 23 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:03.563977  9445 raft_consensus.cc:3060] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.568210  8640 catalog_manager.cc:5697] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 20 to 21, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 21 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:03.571334 16616 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:03.572055  8640 catalog_manager.cc:5697] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 21 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:03.575090  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:03:03.582430  8789 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.590719  9448 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:03:03.596418  9445 raft_consensus.cc:2468] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:03:03.628190 16657 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:03.650887  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): ChangeConfig:REMOVE_PEER RPC failed for tablet 31c225a50048451db8964738c5db7540: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:03.651356  8863 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:03.656615  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:03.663131  8640 catalog_manager.cc:5697] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 24 to 25, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 25 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.678699 16657 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:03.691596  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:03.692317  8898 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 16.
I20260709 13:03:03.720157  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9402cffd3f284d28bf964bd56144a185" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:03:03.720755  9445 raft_consensus.cc:3060] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:03.721062  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.722898 16665 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:03.723274 16665 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b60591217ee24378bb66a86d6d0dda52)
I20260709 13:03:03.723641 16665 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:03.737124  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:03.738114  9445 raft_consensus.cc:2468] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 20.
W20260709 13:03:03.747206  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.763388 16665 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.764909  8862 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.766062 16677 raft_consensus.cc:2804] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:03.766211 16665 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Requested vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:03.766508  8862 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.766616 16677 raft_consensus.cc:697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
W20260709 13:03:03.767005  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:03.767611 16677 consensus_queue.cc:237] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.770061 16643 raft_consensus.cc:2804] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:03.770716 16643 raft_consensus.cc:697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:03:03.771704  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:03:03.771596 16643 consensus_queue.cc:237] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.772269  9445 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:03.774856  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "29c7934ee528454785432732d01ed0a0" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:03:03.775494  8895 raft_consensus.cc:3060] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:03.782859  9445 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
W20260709 13:03:03.785662  8789 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:03.787369  8895 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7934ee528454785432732d01ed0a0 in term 17.
I20260709 13:03:03.788640  8787 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2; no voters: b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:03.789652 16665 raft_consensus.cc:2804] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:03.790205 16665 raft_consensus.cc:697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Becoming Leader. State: Replica: 29c7934ee528454785432732d01ed0a0, State: Running, Role: LEADER
I20260709 13:03:03.793018 16665 consensus_queue.cc:237] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:03.804975  8640 catalog_manager.cc:5697] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 21 to 22, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:03.809302  7638 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:03.811519  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.812867  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:03.827818  8640 catalog_manager.cc:5697] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 19 to 20, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 20 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.835670  8647 catalog_manager.cc:5697] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 reported cstate change: term changed from 16 to 17, leader changed from b60591217ee24378bb66a86d6d0dda52 (127.7.117.129) to 29c7934ee528454785432732d01ed0a0 (127.7.117.130). New cstate: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.845582  8824 raft_consensus.cc:1275] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:03:03.854872  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.865599 16677 consensus_queue.cc:1048] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:03.919458  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:03.922436  8822 raft_consensus.cc:1275] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:03.924706 16643 consensus_queue.cc:1048] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:03.987064  8895 raft_consensus.cc:1275] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:03:03.987730  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.989226 16660 consensus_queue.cc:1048] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:04.007318  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:04.035166  8824 raft_consensus.cc:1275] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:03:04.039386 16564 consensus_queue.cc:1048] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:03:04.047585  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800284a60 after lost signal to thread 7641
W20260709 13:03:04.049820  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08003100e0 after lost signal to thread 8681
W20260709 13:03:04.051182  8678 debug-util.cc:398] Leaking SignalData structure 0x7b080008a200 after lost signal to thread 8848
I20260709 13:03:04.051587  8822 raft_consensus.cc:1275] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:03:04.052294  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800090880 after lost signal to thread 8922
I20260709 13:03:04.053231 16677 consensus_queue.cc:1048] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:04.056535  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7080 after lost signal to thread 9473
W20260709 13:03:04.082621  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:04.135349  9445 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:03:04.144361 16657 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:03:04.169697  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:04.179755  9445 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:04.181587 16665 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:04.255599  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:04.334951  8824 raft_consensus.cc:1275] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:03:04.413743  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 55348us
I20260709 13:03:04.413854 16695 consensus_queue.cc:1048] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:04.430896  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:04.431717  9448 raft_consensus.cc:1275] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:03:04.443984  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:04.454177  8898 raft_consensus.cc:1275] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Refusing update from remote peer 29c7934ee528454785432732d01ed0a0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:04.468632 16711 consensus_queue.cc:1048] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:03:04.457360 16706 consensus_queue.cc:1048] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
W20260709 13:03:04.507207  9414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111) [suppressed 427 similar messages]
W20260709 13:03:04.513945  8789 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:04.539155  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:04.559355  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.572551  8862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38693, user_credentials={real_user=slave}} blocked reactor thread for 85809.1us
W20260709 13:03:04.573693  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:04.585759  9443 raft_consensus.cc:1275] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:03:04.622078  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.641917  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.659811  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:04.661823 16716 consensus_queue.cc:1048] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:04.696154  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.696725  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.727058  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.735879  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.755487  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.809990  7638 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:04.908391  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.908998  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.933213  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.936491  8633 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 71933us
W20260709 13:03:04.961582  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:04.968508  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:05.026883  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.038269  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.067708  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.072736  8634 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet fe13736604454be3b391b8e946192961 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 29c7934ee528454785432732d01ed0a0 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:05.075145  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.088467  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.243582  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.333848  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.393784  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.394380  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.463053  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.472738  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.583487  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.605640  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.707845  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:05.813364  7638 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:06.063673  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:06.082815  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:06.129873  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.246652  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.366701  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 49199.9us
W20260709 13:03:06.390307  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.392903  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 47675.3us
W20260709 13:03:06.411173  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:06.420835  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:06.443413  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:06.467548  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.526113  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.542758  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:06.555104  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:06.670468  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.813946  7638 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:06.882117  8789 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:06.882740  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:06.884542  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:06.998893  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:07.023545  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.186087  8633 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 59214.9us
W20260709 13:03:07.193548  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:07.195385  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.229140  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.264642  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:07.265632  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:07.342904  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 77319us
W20260709 13:03:07.343662  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.362828  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.379113  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.392906  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.402938  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:07.407033  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.464560  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.481769  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.525563  9414 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 43846.9us
W20260709 13:03:07.495296  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.525511  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 61003.2us
W20260709 13:03:07.529584  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.598269  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.603766  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.606405  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.612371  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.655541  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.692804  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.811643  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.814630  7638 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:07.892230  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.930902  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.947320  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.978627  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:07.985615  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.056901  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.416697  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.464877  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.474627  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.490958  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:08.492789  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:08.737444  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:08.752547  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:08.815145  7638 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:08.932484  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:08.995857  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.012867  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.041613  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:09.047461  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:09.074718  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.187861  8634 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet fe13736604454be3b391b8e946192961 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 29c7934ee528454785432732d01ed0a0 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 17 leader_uuid: "29c7934ee528454785432732d01ed0a0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:09.199368  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.226322  8789 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.293737  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.351987  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.416990  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.434669  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.445436  8864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 57338.1us
W20260709 13:03:09.459815  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.463776  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.512398  8863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111) [suppressed 469 similar messages]
W20260709 13:03:09.616953  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.656333  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.677246  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:09.732327  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:09.771235  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.814913  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.815640  7638 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:09.854492  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.858527  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:09.860901  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.919458  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.942767  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.952070  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:09.991933  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.005963  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.053969  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.096828  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.138484  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.166533  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.281014  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.401661  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.506462  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 51700.9us
W20260709 13:03:10.530649  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.540364  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.586522  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.607733  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.663866  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.715690  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.729715  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.735430  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:10.753259  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:10.825129  7638 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:11.073165  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:11.080008  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:11.127624  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:11.320155  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:11.544942  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:11.548795  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:11.549611  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:11.585824  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:11.644280  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:11.686509  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:11.690862  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:11.825645  7638 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 8770)
W20260709 13:03:11.937752  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:11.975996  8789 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:12.009363  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:12.072618  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.139648  8789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 67084.9us
W20260709 13:03:12.197113  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.240613  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.282359  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08003d71a0 after lost signal to thread 7641
W20260709 13:03:12.283969  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2f80 after lost signal to thread 8681
W20260709 13:03:12.285733  8678 debug-util.cc:398] Leaking SignalData structure 0x7b0800090cc0 after lost signal to thread 8848
W20260709 13:03:12.287128  8678 debug-util.cc:398] Leaking SignalData structure 0x7b080031aca0 after lost signal to thread 8922
W20260709 13:03:12.288580  8678 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab000 after lost signal to thread 9473
W20260709 13:03:12.305639  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.323088  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.250087  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:12.334430  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 84398us
W20260709 13:03:12.361580  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.374799  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.391316  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:12.472153  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.472819  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.517701  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.533713  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.541623  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.588024  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:12.593381  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.598083  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.689952  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.729161  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:12.762540  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.769939  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.784642  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.817919  8789 consensus_peers.cc:597] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.862808  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.956892  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:12.970175  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.100274  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.103343  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.124102  8789 consensus_peers.cc:597] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.279353  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.286522  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.361413  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.464244  8789 consensus_peers.cc:597] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.610158  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:13.741267  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:13.758296  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.824784  8789 consensus_peers.cc:597] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:13.907531  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:13.977193  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.012296  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:14.138425  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:14.176520  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.341126  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:14.374239  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.439177  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.477525  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:14.497970  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:14.516582  8863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111) [suppressed 453 similar messages]
W20260709 13:03:14.526836  8789 consensus_peers.cc:597] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.542584  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:14.566849  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
W20260709 13:03:14.580497  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.668130  8789 consensus_peers.cc:597] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:14.665478  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:14.746088  8789 consensus_peers.cc:597] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.875010  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.893348  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.928988  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:14.963357  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:14.987547  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
W20260709 13:03:15.021816  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.033722  8864 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.049638  8864 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.055117  8789 consensus_peers.cc:597] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.059921  8864 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.073670  8789 consensus_peers.cc:597] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.088833  8789 consensus_peers.cc:597] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.089596  8864 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.104507  9415 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.112666  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.116576  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:15.117684  7638 tablet_replica.cc:333] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.118818  7638 raft_consensus.cc:2243] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.120222  7638 raft_consensus.cc:2272] T c6ea7c9c3095497eb93f5d5d412ad667 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.123517  7638 tablet_replica.cc:333] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.133114  8864 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.134194  7638 raft_consensus.cc:2243] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:15.133819  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.136411  7638 raft_consensus.cc:2272] T 31c225a50048451db8964738c5db7540 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.137451  9415 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.139649  7638 tablet_replica.cc:333] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.140637  7638 raft_consensus.cc:2243] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.142731  7638 raft_consensus.cc:2272] T 3006e1c3fb5b4c66852cbc955265f1cb P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.146023  7638 tablet_replica.cc:333] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.147081  7638 raft_consensus.cc:2243] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.148276  7638 raft_consensus.cc:2272] T 7884996e8b84433f80124e36111f58ac P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.152065  9415 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.152971  7638 tablet_replica.cc:333] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.158529  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:15.155162  7638 raft_consensus.cc:2243] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:15.162248  8864 consensus_peers.cc:597] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.165534  8864 consensus_peers.cc:597] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.168054  7638 raft_consensus.cc:2272] T 265e5f9784a6426482f5948fbfcc995c P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.176147  8864 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.179266  7638 tablet_replica.cc:333] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.180851  7638 raft_consensus.cc:2243] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.182772  7638 raft_consensus.cc:2272] T 0192ad465a724ab581b9529345f73bba P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.187225  7638 tablet_replica.cc:333] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.188200  7638 raft_consensus.cc:2243] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.189239  7638 raft_consensus.cc:2272] T 784f5ebc699a4812aaa4be775e13350a P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.196766  7638 tablet_replica.cc:333] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.197768  7638 raft_consensus.cc:2243] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.200117  7638 raft_consensus.cc:2272] T b3894b893d1341819b9b2b487f3b2b26 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.212303  9415 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.213451  8864 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.214059  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.214176  8864 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.217113  7638 tablet_replica.cc:333] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.218146  7638 raft_consensus.cc:2243] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.218802  7638 raft_consensus.cc:2272] T e29b7d4482f943ce897e5a8d5bbb48c4 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.221954  7638 tablet_replica.cc:333] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.229460  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.234459  9415 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.222956  7638 raft_consensus.cc:2243] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.242530  7638 raft_consensus.cc:2272] T f1a7b40e33c24527acba97c95cee9272 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.246567  7638 tablet_replica.cc:333] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.247606  7638 raft_consensus.cc:2243] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.248293  7638 raft_consensus.cc:2272] T 4ac343934349411088cbf7a05932c2a5 P 29c7934ee528454785432732d01ed0a0 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.251102  7638 tablet_replica.cc:333] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.251968  7638 raft_consensus.cc:2243] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.252727  7638 raft_consensus.cc:2272] T 83a998af2b1141848fa02e188ffcd3c6 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.263960  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.265452  7638 tablet_replica.cc:333] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.266384  7638 raft_consensus.cc:2243] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.267056  7638 raft_consensus.cc:2272] T 739742728baa463f9d9f5e5884dd2e30 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.267131  8789 consensus_peers.cc:597] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.272766  7638 tablet_replica.cc:333] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.274017  8864 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.279049  7638 raft_consensus.cc:2243] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.280063  7638 raft_consensus.cc:2272] T d4f573b86fda4b3e8fd982da9615dc8e P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.285859  7638 tablet_replica.cc:333] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.288606  7638 raft_consensus.cc:2243] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.290392  7638 raft_consensus.cc:2272] T 0eb392189d54431c863a3b787fb95779 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.302598  8864 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.308643  9415 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.324169  8864 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.325680  9415 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.343868  8864 consensus_peers.cc:597] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.350617  9415 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.351285  8789 consensus_peers.cc:597] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.351435  7638 tablet_replica.cc:333] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.353741  7638 raft_consensus.cc:2243] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:03:15.354765  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:15.360929  7638 raft_consensus.cc:2272] T db96d769069547e38edb50dbebdbe00a P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.368499  9415 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.366809  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.371665  8864 consensus_peers.cc:597] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.385241  8864 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.394316  7638 tablet_replica.cc:333] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.397689  8864 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.398962  7638 raft_consensus.cc:2243] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.399827  7638 raft_consensus.cc:2272] T dada29d2380a453682f5dfef36dcf407 P 29c7934ee528454785432732d01ed0a0 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.404152  7638 tablet_replica.cc:333] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.408125  7638 raft_consensus.cc:2243] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.414717  7638 raft_consensus.cc:2272] T 90645da839314a85a4bdd72950c292b9 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.421669  9415 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.425901  7638 tablet_replica.cc:333] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.428290  7638 raft_consensus.cc:2243] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.428988  7638 raft_consensus.cc:2272] T 33a0adf8736244d3b208cbc85700f0f9 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.432377  7638 tablet_replica.cc:333] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.442080  7638 raft_consensus.cc:2243] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.442869  7638 raft_consensus.cc:2272] T 42d038e54cdc4dfaa0cfec37a43480ee P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.458808  7638 tablet_replica.cc:333] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.469951  7638 raft_consensus.cc:2243] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:15.471390  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.471978  7638 raft_consensus.cc:2272] T de733f99bf86451da507373e7f4b78aa P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.475760  7638 tablet_replica.cc:333] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.485987  7638 raft_consensus.cc:2243] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:15.488318  9415 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.493377  7638 raft_consensus.cc:2272] T 4bb4c86bcc704ac2add360ac3e5823df P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.495119  8864 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.496325  7638 tablet_replica.cc:333] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.501803  7638 raft_consensus.cc:2243] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.503028  7638 raft_consensus.cc:2272] T eac82cfba1d244139d9b06d8a6c88635 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.510907  7638 tablet_replica.cc:333] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.511688  7638 raft_consensus.cc:2243] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.512508  7638 raft_consensus.cc:2272] T 932d7bc525ba4af99f7d6b52fc1d7337 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.515139  7638 tablet_replica.cc:333] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.516103  7638 raft_consensus.cc:2243] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.517122  7638 raft_consensus.cc:2272] T c5bf6ec0e6a745d0967928a3361eb8d8 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.520054  7638 tablet_replica.cc:333] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.520958  7638 raft_consensus.cc:2243] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.521720  7638 raft_consensus.cc:2272] T 029b57162d72422bba468c7d86b67fca P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.524430  7638 tablet_replica.cc:333] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.525279  7638 raft_consensus.cc:2243] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.534220  7638 raft_consensus.cc:2272] T 011f1e8a4b474eddb6b217393cbdd488 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.537135  7638 tablet_replica.cc:333] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.545990  7638 raft_consensus.cc:2243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:03:15.540234  9415 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.555723  8864 consensus_peers.cc:597] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.557898  7638 raft_consensus.cc:2272] T 48fca36d2cac4f99b1a750bcc17b5d12 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.560894  7638 tablet_replica.cc:333] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.564031  7638 raft_consensus.cc:2243] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.565629  7638 raft_consensus.cc:2272] T 318340b5bf3b4640972d271299414f94 P 29c7934ee528454785432732d01ed0a0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.587710  7638 tablet_replica.cc:333] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.588652  7638 raft_consensus.cc:2243] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:03:15.588967  9415 consensus_peers.cc:597] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.590049  7638 raft_consensus.cc:2272] T 53b2ea39860b462cb7b87b739253a0c9 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.593142  7638 tablet_replica.cc:333] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.594018  7638 raft_consensus.cc:2243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.596662  7638 raft_consensus.cc:2272] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 29c7934ee528454785432732d01ed0a0 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.605968  8864 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.636830  7638 tablet_replica.cc:333] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.642948  7638 raft_consensus.cc:2243] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:15.643006  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:15.643970  7638 raft_consensus.cc:2272] T b476fe65810048fdb66dbeff393c8ccb P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.647312  7638 tablet_replica.cc:333] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.658205  7638 raft_consensus.cc:2243] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.659171  7638 raft_consensus.cc:2272] T 53864dd6ef544420915d557a63d5cd91 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.663884  7638 tablet_replica.cc:333] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.664970  7638 raft_consensus.cc:2243] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.665867  7638 raft_consensus.cc:2272] T f091122e10604f27acd758d1f8da8081 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.669328  7638 tablet_replica.cc:333] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.678921  7638 raft_consensus.cc:2243] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.679730  7638 raft_consensus.cc:2272] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.683640  7638 tablet_replica.cc:333] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.693239  7638 raft_consensus.cc:2243] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.694559  7638 raft_consensus.cc:2272] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.697939  7638 tablet_replica.cc:333] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
W20260709 13:03:15.698402  9415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 43602.6us
I20260709 13:03:15.698915  7638 raft_consensus.cc:2243] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.710153  7638 raft_consensus.cc:2272] T cb3dbb5cac024c52a371cbe4f0f1e952 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.713145  7638 tablet_replica.cc:333] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.714016  7638 raft_consensus.cc:2243] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.714860  7638 raft_consensus.cc:2272] T c24c6743859d43109fabeaf7ee8c8149 P 29c7934ee528454785432732d01ed0a0 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.718325  7638 tablet_replica.cc:333] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.719229  7638 raft_consensus.cc:2243] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.719977  7638 raft_consensus.cc:2272] T 6fa55bf932444ccebb2b7f6583ead84f P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.723372  7638 tablet_replica.cc:333] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.733994  7638 raft_consensus.cc:2243] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.734783  7638 raft_consensus.cc:2272] T c782391e02e74664be7b5f81a3a9478d P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.746327  7638 tablet_replica.cc:333] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.747252  7638 raft_consensus.cc:2243] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.748237  7638 raft_consensus.cc:2272] T 108cbfcea5a2457d89f3cf70cf52b3ad P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.751209  7638 tablet_replica.cc:333] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.752077  7638 raft_consensus.cc:2243] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.759321  7638 raft_consensus.cc:2272] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.762585  7638 tablet_replica.cc:333] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.763489  7638 raft_consensus.cc:2243] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.764235  7638 raft_consensus.cc:2272] T 1f2e6f6b24e84190b722a8e69bbe97fa P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.778573  7638 tablet_replica.cc:333] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.779803  7638 raft_consensus.cc:2243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:15.780911  7638 raft_consensus.cc:2272] T 234e3fa2a66f42f3a5a755f5f4adc858 P 29c7934ee528454785432732d01ed0a0 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.783794  7638 tablet_replica.cc:333] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.784729  7638 raft_consensus.cc:2243] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.786909  7638 raft_consensus.cc:2272] T fe13736604454be3b391b8e946192961 P 29c7934ee528454785432732d01ed0a0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.799065  7638 tablet_replica.cc:333] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.800141  7638 raft_consensus.cc:2243] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.801584  7638 raft_consensus.cc:2272] T 4c91581ed67848b791865dfe11e4fc78 P 29c7934ee528454785432732d01ed0a0 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.804639  7638 tablet_replica.cc:333] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.824548  7638 raft_consensus.cc:2243] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.831151  7638 raft_consensus.cc:2272] T d3e6b55229c842abb4b6a8551191af12 P 29c7934ee528454785432732d01ed0a0 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:15.833951  7638 tablet_replica.cc:333] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0: stopping tablet replica
I20260709 13:03:15.834697  7638 raft_consensus.cc:2243] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:15.842835  7638 raft_consensus.cc:2272] T 44b3244bf12f42aba79421575a96c621 P 29c7934ee528454785432732d01ed0a0 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:15.853384  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.874176  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.965178  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:15.983685  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:16.023697  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:16.202998  9415 connection.cc:769] client connection to 127.7.117.130:44185 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:03:16.239869  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:16.302134  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:16.328402  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:16.477798 16765 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.478468 16765 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.480643 16765 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.488870  8863 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:16.491653  8864 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.492182  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.493016 16765 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:16.551039  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:16.552084  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:16.577581 16766 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.578203 16766 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.580415 16766 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.592895  9415 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.594185  9414 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.594647  9414 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.595871 16766 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:16.617993 16766 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.618585 16766 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.620836 16766 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.629725  9414 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:16.630448  9415 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.630860  9415 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.631031 16765 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.632977 16765 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.631717 16744 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:16.637751 16763 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.641253 16763 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.643821 16763 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:16.645510  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:16.647651 16765 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.640313  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:16.649025  9415 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.660382  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:16.662889 16766 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:16.667865 16765 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.668578 16765 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.671061 16765 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.673638  8864 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.690452  8863 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.691978  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.700762 16773 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:16.702111  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:16.702647  8864 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.714716  8863 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.723089  8863 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.729854 16769 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:16.731957 16773 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.732503 16773 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.734948 16773 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:16.739060 16773 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.739630 16773 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.743139 16773 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:16.744118  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
W20260709 13:03:16.748449  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:16.780730  8863 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:16.788457  8864 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.789232  8864 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.791962 16773 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:16.793219  8864 leader_election.cc:336] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.793743  8864 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:16.794485 16773 raft_consensus.cc:2749] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:16.798754  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:16.830521 16769 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.831087 16769 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.833756 16773 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.834369 16773 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.836563 16773 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:16.837226 16769 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:16.839052  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:16.839972  9445 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:03:16.841620  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:03:16.841847  8863 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.842770 16769 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:16.843221 16769 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.843650 16769 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:16.845894 16773 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.846526 16773 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.848984 16773 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.855554  8863 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.857573 16763 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.860554 16763 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.862890 16763 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:16.865551  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:16.866312  8895 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 19.
I20260709 13:03:16.871389  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
W20260709 13:03:16.872644  9415 leader_election.cc:336] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.873109 16769 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.872720 16766 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:16.874100 16766 raft_consensus.cc:493] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.874557 16766 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:16.876006  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:16.881422  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db96d769069547e38edb50dbebdbe00a" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071"
I20260709 13:03:16.882385  9445 raft_consensus.cc:3060] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:16.890560  9445 raft_consensus.cc:2468] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 22.
I20260709 13:03:16.891253 16766 raft_consensus.cc:515] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.892009  8862 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 
I20260709 13:03:16.892871 16773 raft_consensus.cc:2804] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:16.893530 16773 raft_consensus.cc:697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Becoming Leader. State: Replica: 7fd110c312c14e17a2bbe6d453493cf2, State: Running, Role: LEADER
I20260709 13:03:16.894579 16773 consensus_queue.cc:237] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:16.899370  9415 leader_election.cc:336] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.882040  8864 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.910360  8864 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.913129  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.911792 16769 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:16.918594 16765 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:16.920004  8864 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.922690  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.923977 16765 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:16.934998  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:16.937819  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:16.943209  8864 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:16.949255 16773 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.949801 16773 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:16.949743  8640 catalog_manager.cc:5697] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 reported cstate change: term changed from 21 to 22, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131). New cstate: current_term: 22 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.953851 16766 leader_election.cc:290] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:16.956670  8863 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.957549 16773 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:16.958750  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fca36d2cac4f99b1a750bcc17b5d12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:03:16.959323  8895 raft_consensus.cc:3060] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:16.962131 16766 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:16.962670 16766 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.967725  8895 raft_consensus.cc:2468] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
W20260709 13:03:16.974052  9415 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:16.976754  9414 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:16.977519  9414 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:16.979526 16763 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:16.980970  9412 leader_election.cc:304] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:03:16.981808 16763 raft_consensus.cc:2804] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:16.982282 16763 raft_consensus.cc:697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:03:16.983165 16763 consensus_queue.cc:237] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:16.986312 16766 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:16.996832  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:16.999577 16773 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.000276 16773 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:17.005405 16769 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.006201 16769 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:17.008518 16769 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:17.013403  8863 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:17.014353 16773 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:16.996814  8864 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.016108  8864 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:17.021485 16773 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:17.029853  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 45053.1us
I20260709 13:03:17.035652  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:17.036610  8640 catalog_manager.cc:5697] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 19 to 20, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 20 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.066970  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:17.082630  8864 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.083132  8864 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 112105225cc140459688be5d31dcb071, 29c7934ee528454785432732d01ed0a0
I20260709 13:03:17.083922 16769 raft_consensus.cc:2749] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:17.085238  8864 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.085713  8864 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:17.086453 16769 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:17.110013 16766 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.110666 16766 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:17.124805  9415 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:17.126516  9414 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:17.126946  9414 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:17.127681 16763 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:17.128345 16766 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:17.130399  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
W20260709 13:03:17.191562  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:17.202136 16773 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.205667 16773 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:17.218034  7638 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8845)
I20260709 13:03:17.227860 16765 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.228799 16765 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:17.232779 16765 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:17.234423 16773 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:17.235663  8863 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:17.243189  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:17.243886  8864 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.245194  8864 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:17.252308 16782 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:17.282174 16763 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.283188 16763 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:17.287375  8864 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.289108 16763 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:17.293988  9415 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.296129  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:17.296577  9448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:17.297461  9445 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:03:17.297739  9448 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:03:17.302570  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:17.304852 16783 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:17.315745  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:17.316690  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 20.
W20260709 13:03:17.319259  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:17.329003  9412 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:03:17.332258 16778 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:17.332653 16778 raft_consensus.cc:493] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:17.332993 16778 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:17.350795  9445 raft_consensus.cc:1275] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Refusing update from remote peer 7fd110c312c14e17a2bbe6d453493cf2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:17.361977 16785 consensus_queue.cc:1048] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:17.364334  8864 consensus_peers.cc:597] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:17.410261 16778 raft_consensus.cc:515] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:17.414345  8864 consensus_peers.cc:597] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:17.420727  8864 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:17.421449 16778 leader_election.cc:290] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Requested vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:17.423043  8898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3e6b55229c842abb4b6a8551191af12" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2"
I20260709 13:03:17.423662  8898 raft_consensus.cc:3060] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:17.443364  9415 leader_election.cc:336] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.452448  8898 raft_consensus.cc:2468] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 21.
W20260709 13:03:17.475317  8864 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.494607  9415 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.497184  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:17.498193  9412 leader_election.cc:304] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0
I20260709 13:03:17.501410 16778 raft_consensus.cc:2804] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:17.505520 16778 raft_consensus.cc:697] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Becoming Leader. State: Replica: 112105225cc140459688be5d31dcb071, State: Running, Role: LEADER
I20260709 13:03:17.515019 16778 consensus_queue.cc:237] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:17.522477  9415 consensus_peers.cc:597] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:17.524224  8889 raft_consensus.cc:1275] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:03:17.531232  8634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.117.130:44185, user_credentials={real_user=slave}} blocked reactor thread for 70378us
I20260709 13:03:17.541134 16763 consensus_queue.cc:1048] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:17.564800  8634 catalog_manager.cc:4755] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): ChangeConfig:REMOVE_PEER RPC failed for tablet fe13736604454be3b391b8e946192961: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:17.584997  8640 catalog_manager.cc:5697] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 reported cstate change: term changed from 20 to 21, leader changed from 29c7934ee528454785432732d01ed0a0 (127.7.117.130) to 112105225cc140459688be5d31dcb071 (127.7.117.132). New cstate: current_term: 21 leader_uuid: "112105225cc140459688be5d31dcb071" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:17.602741  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.618335  9415 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.624272  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.625012  8864 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.628958  9415 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.634562  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.639135  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.654887  9415 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.665351  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:17.695204  8864 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.696770  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:17.700081  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.700656  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.769891  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.771917  9415 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.777944  8864 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.778630  8864 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.788141  9415 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.802809  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.808954  9415 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.830596  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.847012  9415 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.852878  8864 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.862986  8864 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.875454  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.876055  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:17.889585  9415 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.890435  8864 consensus_peers.cc:597] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.890951  8864 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.991844  9415 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.060844  8864 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.082369  9415 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:18.104341  8898 raft_consensus.cc:1275] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Refusing update from remote peer 112105225cc140459688be5d31dcb071: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:03:18.121388  8631 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 31c225a50048451db8964738c5db7540 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 7fd110c312c14e17a2bbe6d453493cf2 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 25 leader_uuid: "7fd110c312c14e17a2bbe6d453493cf2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } } }
W20260709 13:03:18.120335  9415 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.136423  8864 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.159822  9415 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:18.163370 16744 consensus_queue.cc:1048] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:03:18.220738  7638 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 8845)
W20260709 13:03:18.226528  8864 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.258440  9415 consensus_peers.cc:597] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.282747  8864 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.285106  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:18.292810  8864 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:18.311985 16796 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.312716 16796 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.320269 16796 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.321369  8864 consensus_peers.cc:597] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.354038  8863 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:18.380369 16796 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.381067 16796 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.383948 16796 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
I20260709 13:03:18.391952 16766 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.392742 16766 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:18.395777 16766 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.399941  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45382.9us
I20260709 13:03:18.404346 16766 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.405368 16766 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:18.407642 16766 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.414517  8864 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.415287  8864 consensus_peers.cc:597] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.448148  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:18.449851  9414 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:18.450263  9415 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.451184  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:18.453984  8863 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:18.458082  9415 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.458510 16766 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:18.499945  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 45403.5us
W20260709 13:03:18.520502  9414 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:18.521066  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:18.521832 16766 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:18.526844 16796 raft_consensus.cc:493] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.527521 16796 raft_consensus.cc:515] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.529816 16796 leader_election.cc:290] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.543972  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:18.544723  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:18.560715  8864 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.561170  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:18.562880 16796 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:18.564554  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:18.579751  8863 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:18.602422  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:18.603480 16796 raft_consensus.cc:493] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.604254 16796 raft_consensus.cc:515] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.607671 16796 leader_election.cc:290] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.608316  9412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:44123, user_credentials={real_user=slave}} blocked reactor thread for 55126.8us
W20260709 13:03:18.635393  8864 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.635939  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:18.636929  8864 consensus_peers.cc:597] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:18.637050 16796 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:18.637687  8864 leader_election.cc:336] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.638114  8864 leader_election.cc:304] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:18.640257 16783 raft_consensus.cc:2749] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:18.649518 16796 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.650278 16796 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.652403 16796 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.652467  8864 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:18.660925  8863 leader_election.cc:336] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:18.666961  8864 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.699210  8863 leader_election.cc:304] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:18.707681  8863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:39991, user_credentials={real_user=slave}} blocked reactor thread for 46793.4us
I20260709 13:03:18.708384 16796 raft_consensus.cc:2749] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:18.713627 16796 raft_consensus.cc:493] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:18.725153 16796 raft_consensus.cc:515] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:18.727517 16796 leader_election.cc:290] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:18.730082  8863 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:18.732005  8864 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:18.731698  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:18.738018 16796 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:18.775308  8863 leader_election.cc:336] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:18.779336  8863 leader_election.cc:304] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:18.780196 16784 raft_consensus.cc:2749] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:18.934029  8863 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.020083  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.078677  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:19.142705 16783 raft_consensus.cc:493] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:19.143330 16783 raft_consensus.cc:515] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:19.145435 16783 leader_election.cc:290] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:19.153824  8863 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:19.158589  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:19.162611  8863 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:19.192660  8864 leader_election.cc:336] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:19.193238  8864 leader_election.cc:304] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:19.194120 16796 raft_consensus.cc:2749] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:19.221205  7638 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 8845)
W20260709 13:03:19.248013  8863 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.278407  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:19.287492 16784 raft_consensus.cc:493] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:19.288169 16784 raft_consensus.cc:515] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:03:19.303776  8863 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:19.304739 16783 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:19.305337 16783 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:19.308243 16783 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
I20260709 13:03:19.310920  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b3244bf12f42aba79421575a96c621" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:19.314749  9445 raft_consensus.cc:2468] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 21.
I20260709 13:03:19.333617 16784 leader_election.cc:290] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:19.340826  8864 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:19.353157  8864 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:19.385089  8863 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:19.389828  8863 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:19.392755 16783 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:19.408718  8863 leader_election.cc:336] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:19.409456  8863 leader_election.cc:304] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:19.411966 16802 raft_consensus.cc:2749] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:19.422971  8863 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.442209  8863 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:19.459754 16802 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:19.460449 16802 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:19.464541  9445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "7fd110c312c14e17a2bbe6d453493cf2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "112105225cc140459688be5d31dcb071" is_pre_election: true
I20260709 13:03:19.465209  9445 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fd110c312c14e17a2bbe6d453493cf2 in term 17.
I20260709 13:03:19.466853 16802 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 112105225cc140459688be5d31dcb071 (127.7.117.132:38693)
W20260709 13:03:19.504627  8863 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:19.507488  8863 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:19.512538  8863 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:19.523973  8864 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:19.529855  8864 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071, 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:19.529325  8863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111) [suppressed 730 similar messages]
I20260709 13:03:19.535095 16802 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:19.560689  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.581251  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:19.681655  8864 consensus_peers.cc:597] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.737306  8863 consensus_peers.cc:597] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.746757  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:19.755584  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:19.761546  9415 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.769860  9415 consensus_peers.cc:597] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:19.843793  8864 consensus_peers.cc:597] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:19.891469  8864 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.925375  8864 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.948904  8864 consensus_peers.cc:597] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.051975  9415 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.076435  8863 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.085181  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.108033  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.114557  9415 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.131466  8863 consensus_peers.cc:597] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:20.132858  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.143199  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:20.150938  8864 consensus_peers.cc:597] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.153740  9415 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.155458  8864 consensus_peers.cc:597] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:20.189138 16802 raft_consensus.cc:493] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:20.189781 16802 raft_consensus.cc:515] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
W20260709 13:03:20.198962  8863 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:20.200642 16802 leader_election.cc:290] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:20.205220  8864 consensus_peers.cc:597] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.206452  8863 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.213050  8864 leader_election.cc:336] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:20.213488  8864 leader_election.cc:304] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:20.214458  9415 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:20.214618 16802 raft_consensus.cc:2749] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:20.222231  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.232043  8864 consensus_peers.cc:597] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.232556  9415 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.236115  9415 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.258656  8864 consensus_peers.cc:597] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.261498  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.276619  8863 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.292984  8864 consensus_peers.cc:597] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:20.317152 16802 raft_consensus.cc:493] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:20.317961 16802 raft_consensus.cc:515] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:20.321431 16802 leader_election.cc:290] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:20.324232  8863 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:20.336023  8864 leader_election.cc:336] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:20.336532  8864 leader_election.cc:304] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:20.337327 16802 raft_consensus.cc:2749] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:20.352880 16803 raft_consensus.cc:493] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:20.353495 16803 raft_consensus.cc:515] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:20.355646 16803 leader_election.cc:290] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:20.368083  9415 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.369762  9414 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:20.381827  9415 leader_election.cc:336] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:20.384519  9415 leader_election.cc:304] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:20.385450 16803 raft_consensus.cc:2749] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:20.407759  9415 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.426599  9415 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.447909  8863 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.472792  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.475992  8864 consensus_peers.cc:597] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.476501  8864 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.481101  9415 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.496656  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.536903  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.665347  8864 consensus_peers.cc:597] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.668654  8864 consensus_peers.cc:597] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.669247  8864 consensus_peers.cc:597] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:20.717145 16784 raft_consensus.cc:493] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:20.718052 16784 raft_consensus.cc:515] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } }
I20260709 13:03:20.720094 16784 leader_election.cc:290] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:20.729705  8864 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:20.730720  9415 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:20.738148  9415 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.740001  8864 consensus_peers.cc:597] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.755810  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:20.769039  8864 consensus_peers.cc:597] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.776301  8863 leader_election.cc:336] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:20.776906  8863 leader_election.cc:304] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fd110c312c14e17a2bbe6d453493cf2; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:20.777719 16811 raft_consensus.cc:2749] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:20.798593  8864 consensus_peers.cc:597] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.841496  9415 consensus_peers.cc:597] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.848791  8864 consensus_peers.cc:597] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.875164  8864 consensus_peers.cc:597] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:20.936189  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
W20260709 13:03:20.971800  8863 consensus_peers.cc:597] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:21.070930  9414 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:21.087208 16807 raft_consensus.cc:493] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:21.087920 16807 raft_consensus.cc:515] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } attrs { replace: true } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:21.101392  9414 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:21.118184 16807 leader_election.cc:290] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:21.123939  9415 leader_election.cc:336] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:21.124490  9415 leader_election.cc:304] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:21.132212 16809 raft_consensus.cc:2749] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:21.135756  8895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe13736604454be3b391b8e946192961" candidate_uuid: "112105225cc140459688be5d31dcb071" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7fd110c312c14e17a2bbe6d453493cf2" is_pre_election: true
I20260709 13:03:21.136605  8895 raft_consensus.cc:2468] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 112105225cc140459688be5d31dcb071 in term 17.
I20260709 13:03:21.139106  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
W20260709 13:03:21.175181  9414 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:21.187083  9412 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.187809  9412 consensus_peers.cc:597] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.207502  9412 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.208153  9412 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.221698  9412 consensus_peers.cc:597] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.222324  9412 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.223893  9412 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.225998  8863 consensus_peers.cc:597] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:21.259869  9412 consensus_peers.cc:597] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.267419  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:21.268914  7638 tablet_replica.cc:333] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.269945  7638 raft_consensus.cc:2243] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.271308  7638 raft_consensus.cc:2272] T c6ea7c9c3095497eb93f5d5d412ad667 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.274098  7638 tablet_replica.cc:333] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.274987  7638 raft_consensus.cc:2243] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.281332  7638 raft_consensus.cc:2272] T 31c225a50048451db8964738c5db7540 P 7fd110c312c14e17a2bbe6d453493cf2 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.284436  9412 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.288661  7638 tablet_replica.cc:333] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.289662  7638 raft_consensus.cc:2243] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.290465  7638 raft_consensus.cc:2272] T a07b06e72d474091a0bcf7de181b509f P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.292824  9412 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.293207  7638 tablet_replica.cc:333] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.293979  7638 raft_consensus.cc:2243] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.294821  7638 raft_consensus.cc:2272] T dc6dcf2342014ed6b65dd5af29d18d3b P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.296988  7638 tablet_replica.cc:333] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.297739  7638 raft_consensus.cc:2243] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.298346  7638 raft_consensus.cc:2272] T eac82cfba1d244139d9b06d8a6c88635 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.306699  7638 tablet_replica.cc:333] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.307590  7638 raft_consensus.cc:2243] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.308437  7638 raft_consensus.cc:2272] T 0192ad465a724ab581b9529345f73bba P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.311136  7638 tablet_replica.cc:333] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.311972  7638 raft_consensus.cc:2243] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.313437  7638 raft_consensus.cc:2272] T b3894b893d1341819b9b2b487f3b2b26 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.318655  7638 tablet_replica.cc:333] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.319558  7638 raft_consensus.cc:2243] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.320739  7638 raft_consensus.cc:2272] T 784f5ebc699a4812aaa4be775e13350a P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.323305  7638 tablet_replica.cc:333] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.324103  7638 raft_consensus.cc:2243] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.324721  7638 raft_consensus.cc:2272] T f1a7b40e33c24527acba97c95cee9272 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.327382  7638 tablet_replica.cc:333] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.328225  7638 raft_consensus.cc:2243] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.329387  7638 raft_consensus.cc:2272] T 739742728baa463f9d9f5e5884dd2e30 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.333629  8863 consensus_peers.cc:597] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:21.335987  7638 tablet_replica.cc:333] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.336867  7638 raft_consensus.cc:2243] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.338630  7638 raft_consensus.cc:2272] T e29b7d4482f943ce897e5a8d5bbb48c4 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.341444  7638 tablet_replica.cc:333] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.342227  7638 raft_consensus.cc:2243] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.343322  7638 raft_consensus.cc:2272] T 83a998af2b1141848fa02e188ffcd3c6 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.346032  7638 tablet_replica.cc:333] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
W20260709 13:03:21.346051  9412 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:21.346848  9412 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.347091  7638 raft_consensus.cc:2243] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.348194  7638 raft_consensus.cc:2272] T db96d769069547e38edb50dbebdbe00a P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.350410  7638 tablet_replica.cc:333] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.350973  7638 raft_consensus.cc:2243] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.351531  7638 raft_consensus.cc:2272] T dada29d2380a453682f5dfef36dcf407 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.353593  7638 tablet_replica.cc:333] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.354224  7638 raft_consensus.cc:2243] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.355016  7638 raft_consensus.cc:2272] T d8087b240e9e4f5d9ea874980d0d7efa P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.362214  7638 tablet_replica.cc:333] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.363188  7638 raft_consensus.cc:2243] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.364588  7638 raft_consensus.cc:2272] T 90645da839314a85a4bdd72950c292b9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.367444  7638 tablet_replica.cc:333] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.368273  7638 raft_consensus.cc:2243] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.369612  7638 raft_consensus.cc:2272] T b476fe65810048fdb66dbeff393c8ccb P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.371680  7638 tablet_replica.cc:333] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.372293  7638 raft_consensus.cc:2243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.373813  7638 raft_consensus.cc:2272] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.376324  7638 tablet_replica.cc:333] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.377250  7638 raft_consensus.cc:2243] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:03:21.378008  9412 consensus_peers.cc:597] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.378376  7638 raft_consensus.cc:2272] T cb3dbb5cac024c52a371cbe4f0f1e952 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.380977  7638 tablet_replica.cc:333] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.381876  7638 raft_consensus.cc:2243] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.383154  7638 raft_consensus.cc:2272] T 9662b1b1994a4a29b1ec6da7c3cd5f39 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.388072  7638 tablet_replica.cc:333] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.390048  7638 raft_consensus.cc:2243] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.390655  7638 raft_consensus.cc:2272] T b4d3d159f1de4558ab5e2c0dcfc6f4d1 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.393496  7638 tablet_replica.cc:333] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.394258  7638 raft_consensus.cc:2243] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.394935  7638 raft_consensus.cc:2272] T 011f1e8a4b474eddb6b217393cbdd488 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.397760  7638 tablet_replica.cc:333] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.398742  7638 raft_consensus.cc:2243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.399355  7638 raft_consensus.cc:2272] T 48fca36d2cac4f99b1a750bcc17b5d12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.401893  7638 tablet_replica.cc:333] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.402702  7638 raft_consensus.cc:2243] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.403319  7638 raft_consensus.cc:2272] T 318340b5bf3b4640972d271299414f94 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.406193  7638 tablet_replica.cc:333] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.406986  7638 raft_consensus.cc:2243] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.407867  7638 raft_consensus.cc:2272] T 3f192c6b771043a893d6fd420c9b9e7d P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.411721  7638 tablet_replica.cc:333] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.412679  7638 raft_consensus.cc:2243] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.413858  7638 raft_consensus.cc:2272] T 42d038e54cdc4dfaa0cfec37a43480ee P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.416651  7638 tablet_replica.cc:333] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.417470  7638 raft_consensus.cc:2243] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.418679  7638 raft_consensus.cc:2272] T de733f99bf86451da507373e7f4b78aa P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:21.420475  9412 consensus_peers.cc:597] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.421623  7638 tablet_replica.cc:333] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.422448  7638 raft_consensus.cc:2243] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.423599  7638 raft_consensus.cc:2272] T 33a0adf8736244d3b208cbc85700f0f9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.426438  7638 tablet_replica.cc:333] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.427227  7638 raft_consensus.cc:2243] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:21.427606  9412 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Couldn't send request to peer 7fd110c312c14e17a2bbe6d453493cf2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:21.428017  7638 raft_consensus.cc:2272] T 4bb4c86bcc704ac2add360ac3e5823df P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.430732  7638 tablet_replica.cc:333] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.431524  7638 raft_consensus.cc:2243] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.432601  7638 raft_consensus.cc:2272] T 932d7bc525ba4af99f7d6b52fc1d7337 P 7fd110c312c14e17a2bbe6d453493cf2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.435352  7638 tablet_replica.cc:333] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.436116  7638 raft_consensus.cc:2243] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.436698  7638 raft_consensus.cc:2272] T 53b2ea39860b462cb7b87b739253a0c9 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.439253  7638 tablet_replica.cc:333] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.439970  7638 raft_consensus.cc:2243] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.441134  7638 raft_consensus.cc:2272] T 9402cffd3f284d28bf964bd56144a185 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.443778  7638 tablet_replica.cc:333] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.444459  7638 raft_consensus.cc:2243] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.445159  7638 raft_consensus.cc:2272] T 75401b4ca97a4c1eb75a016ff2124379 P 7fd110c312c14e17a2bbe6d453493cf2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.447781  7638 tablet_replica.cc:333] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.448500  7638 raft_consensus.cc:2243] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.449846  7638 raft_consensus.cc:2272] T f091122e10604f27acd758d1f8da8081 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.452425  7638 tablet_replica.cc:333] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.453230  7638 raft_consensus.cc:2243] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.454094  7638 raft_consensus.cc:2272] T 1a0644aef66444b58e8633da6993535c P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.456832  7638 tablet_replica.cc:333] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.457536  7638 raft_consensus.cc:2243] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.458894  7638 raft_consensus.cc:2272] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.461365  7638 tablet_replica.cc:333] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.461915  7638 raft_consensus.cc:2243] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.462970  7638 raft_consensus.cc:2272] T c782391e02e74664be7b5f81a3a9478d P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.465463  7638 tablet_replica.cc:333] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.466260  7638 raft_consensus.cc:2243] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.467566  7638 raft_consensus.cc:2272] T 108cbfcea5a2457d89f3cf70cf52b3ad P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.474936  7638 tablet_replica.cc:333] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.475809  7638 raft_consensus.cc:2243] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.477017  7638 raft_consensus.cc:2272] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.479947  7638 tablet_replica.cc:333] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.480631  7638 raft_consensus.cc:2243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.481220  7638 raft_consensus.cc:2272] T 234e3fa2a66f42f3a5a755f5f4adc858 P 7fd110c312c14e17a2bbe6d453493cf2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.484221  7638 tablet_replica.cc:333] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.485038  7638 raft_consensus.cc:2243] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.485757  7638 raft_consensus.cc:2272] T fd126387f2fb45f98d7ca0807566c688 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.488593  7638 tablet_replica.cc:333] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.489568  7638 raft_consensus.cc:2243] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.490300  7638 raft_consensus.cc:2272] T fc45d3a2731b49daacb16548dc742815 P 7fd110c312c14e17a2bbe6d453493cf2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.493172  7638 tablet_replica.cc:333] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.493829  7638 raft_consensus.cc:2243] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.494438  7638 raft_consensus.cc:2272] T 011b370821004185a003be210a525c01 P 7fd110c312c14e17a2bbe6d453493cf2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.497229  7638 tablet_replica.cc:333] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.497973  7638 raft_consensus.cc:2243] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.499135  7638 raft_consensus.cc:2272] T d3e6b55229c842abb4b6a8551191af12 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.501935  7638 tablet_replica.cc:333] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.502758  7638 raft_consensus.cc:2243] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.503841  7638 raft_consensus.cc:2272] T 44b3244bf12f42aba79421575a96c621 P 7fd110c312c14e17a2bbe6d453493cf2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.506801  7638 tablet_replica.cc:333] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.507449  7638 raft_consensus.cc:2243] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.508093  7638 raft_consensus.cc:2272] T 4c91581ed67848b791865dfe11e4fc78 P 7fd110c312c14e17a2bbe6d453493cf2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.510864  7638 tablet_replica.cc:333] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.511611  7638 raft_consensus.cc:2243] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:21.512810  7638 raft_consensus.cc:2272] T c24c6743859d43109fabeaf7ee8c8149 P 7fd110c312c14e17a2bbe6d453493cf2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.515651  7638 tablet_replica.cc:333] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2: stopping tablet replica
I20260709 13:03:21.516347  7638 raft_consensus.cc:2243] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.516992  7638 raft_consensus.cc:2272] T fe13736604454be3b391b8e946192961 P 7fd110c312c14e17a2bbe6d453493cf2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.665846 16807 raft_consensus.cc:493] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:21.666460 16807 raft_consensus.cc:515] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:21.668923 16807 leader_election.cc:290] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185)
W20260709 13:03:21.671566  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003a0ea0 after lost signal to thread 7641
W20260709 13:03:21.673030  9470 debug-util.cc:398] Leaking SignalData structure 0x7b0800387c80 after lost signal to thread 8681
W20260709 13:03:21.674815  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003dda40 after lost signal to thread 8922
W20260709 13:03:21.678220  9415 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:21.678476  9470 debug-util.cc:398] Leaking SignalData structure 0x7b08003eede0 after lost signal to thread 9473
W20260709 13:03:21.678795  9414 leader_election.cc:336] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:21.679193  9414 leader_election.cc:304] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:21.680694 16814 raft_consensus.cc:2749] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:21.700850  9414 consensus_peers.cc:597] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:21.716917  9414 consensus_peers.cc:597] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:21.749756  9414 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:21.939857  9414 consensus_peers.cc:597] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:22.238358 16807 raft_consensus.cc:493] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 29c7934ee528454785432732d01ed0a0)
I20260709 13:03:22.239221 16807 raft_consensus.cc:515] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.243333 16807 leader_election.cc:290] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.252485  9415 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.265718  9415 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.273940  9414 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:22.274555  9414 leader_election.cc:304] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, b60591217ee24378bb66a86d6d0dda52
W20260709 13:03:22.275349  9412 leader_election.cc:336] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.276206 16807 raft_consensus.cc:2749] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:22.313761 16807 raft_consensus.cc:493] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.314390 16807 raft_consensus.cc:515] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } attrs { replace: true } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:22.324028  9415 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:22.325618 16807 leader_election.cc:290] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:22.327791 16814 raft_consensus.cc:493] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.328383 16814 raft_consensus.cc:515] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.330432 16814 leader_election.cc:290] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.353896  9412 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.354378  9412 leader_election.cc:304] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.355300 16807 raft_consensus.cc:2749] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:22.356046  9415 leader_election.cc:336] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.360597  9414 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:22.361047  9414 consensus_peers.cc:597] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:22.361438  9414 leader_election.cc:336] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:22.373451  9412 leader_election.cc:336] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.373919  9412 leader_election.cc:304] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.379458 16807 raft_consensus.cc:2749] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:22.391016  9415 consensus_peers.cc:597] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.408586  9415 consensus_peers.cc:597] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:22.410408  9414 consensus_peers.cc:597] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:22.539523 16807 raft_consensus.cc:493] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.540097 16807 raft_consensus.cc:515] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.543164 16807 leader_election.cc:290] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.559556  9414 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:22.580098  9415 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.594825  9412 leader_election.cc:336] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.595292  9412 leader_election.cc:304] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:22.596392 16807 raft_consensus.cc:2749] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:22.600255  9415 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:22.624703 16807 raft_consensus.cc:493] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.625238 16807 raft_consensus.cc:515] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.629056 16814 raft_consensus.cc:493] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.629633 16814 raft_consensus.cc:515] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.631624 16814 leader_election.cc:290] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:22.628216 16807 leader_election.cc:290] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.638700  9415 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.639176  9415 leader_election.cc:336] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.649235  9415 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.658789  9412 leader_election.cc:336] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.659243  9412 leader_election.cc:304] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
W20260709 13:03:22.660079  9412 leader_election.cc:336] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.660455  9412 leader_election.cc:304] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.661105 16814 raft_consensus.cc:2749] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:22.661780 16814 raft_consensus.cc:2749] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:22.681924  9414 consensus_peers.cc:597] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:22.683241 16807 raft_consensus.cc:493] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.684475 16807 raft_consensus.cc:515] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:22.693710  9415 leader_election.cc:336] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.694456  9415 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:22.697728 16807 leader_election.cc:290] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.710165  9412 leader_election.cc:336] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.710742  9412 leader_election.cc:304] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.711555 16807 raft_consensus.cc:2749] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:22.725610  9414 consensus_peers.cc:597] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:22.749464 16807 raft_consensus.cc:493] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.750177 16807 raft_consensus.cc:515] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.752480 16807 leader_election.cc:290] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.765273  9415 leader_election.cc:336] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:22.769152  9412 leader_election.cc:336] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.769691  9412 leader_election.cc:304] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.770535 16807 raft_consensus.cc:2749] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:22.778960  9415 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:22.783973 16814 raft_consensus.cc:493] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.784605 16814 raft_consensus.cc:515] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b60591217ee24378bb66a86d6d0dda52" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 39991 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
I20260709 13:03:22.786813 16814 leader_election.cc:290] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
W20260709 13:03:22.794806  9414 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
W20260709 13:03:22.795529  9414 consensus_peers.cc:597] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:22.800410  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
W20260709 13:03:22.805024  9412 leader_election.cc:336] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
I20260709 13:03:22.805691  9412 leader_election.cc:304] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 7fd110c312c14e17a2bbe6d453493cf2, b60591217ee24378bb66a86d6d0dda52
I20260709 13:03:22.810798 16807 raft_consensus.cc:2749] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:22.828045  9415 consensus_peers.cc:597] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.834620  9414 consensus_peers.cc:597] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:22.858572  9415 consensus_peers.cc:597] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:22.889218 16807 raft_consensus.cc:493] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7fd110c312c14e17a2bbe6d453493cf2)
I20260709 13:03:22.889882 16807 raft_consensus.cc:515] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "29c7934ee528454785432732d01ed0a0" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 44185 } } peers { permanent_uuid: "7fd110c312c14e17a2bbe6d453493cf2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 44123 } } peers { permanent_uuid: "112105225cc140459688be5d31dcb071" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38693 } attrs { promote: false } }
W20260709 13:03:22.899605  9412 leader_election.cc:336] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123): Network error: Client connection negotiation failed: client connection to 127.7.117.131:44123: connect: Connection refused (error 111)
W20260709 13:03:22.903815  9415 leader_election.cc:336] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
I20260709 13:03:22.904886  9415 leader_election.cc:304] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 112105225cc140459688be5d31dcb071; no voters: 29c7934ee528454785432732d01ed0a0, 7fd110c312c14e17a2bbe6d453493cf2
I20260709 13:03:22.904392 16807 leader_election.cc:290] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185), 7fd110c312c14e17a2bbe6d453493cf2 (127.7.117.131:44123)
I20260709 13:03:22.906064 16814 raft_consensus.cc:2749] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:22.948297  9415 consensus_peers.cc:597] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:22.959605  7638 tablet_server.cc:179] TabletServer@127.7.117.132:0 shutting down...
W20260709 13:03:22.959661  9414 consensus_peers.cc:597] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:22.985762  9415 consensus_peers.cc:597] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 -> Peer 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): Couldn't send request to peer 29c7934ee528454785432732d01ed0a0. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:23.029211  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:23.031324  7638 tablet_replica.cc:333] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.032289  7638 raft_consensus.cc:2243] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.033504  7638 raft_consensus.cc:2272] T d3e6b55229c842abb4b6a8551191af12 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.038074  7638 tablet_replica.cc:333] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.039058  7638 raft_consensus.cc:2243] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.039940  7638 raft_consensus.cc:2272] T 44b3244bf12f42aba79421575a96c621 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.044647  7638 tablet_replica.cc:333] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.045579  7638 raft_consensus.cc:2243] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.047559  7638 raft_consensus.cc:2272] T d8087b240e9e4f5d9ea874980d0d7efa P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.050766  7638 tablet_replica.cc:333] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.051694  7638 raft_consensus.cc:2243] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.052953  7638 raft_consensus.cc:2272] T 1f2e6f6b24e84190b722a8e69bbe97fa P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.057152  7638 tablet_replica.cc:333] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.058038  7638 raft_consensus.cc:2243] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.060434  7638 raft_consensus.cc:2272] T 48fca36d2cac4f99b1a750bcc17b5d12 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.063248  7638 tablet_replica.cc:333] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.064117  7638 raft_consensus.cc:2243] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.064841  7638 raft_consensus.cc:2272] T 0eb392189d54431c863a3b787fb95779 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:23.065243  9414 consensus_peers.cc:597] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:23.067668  7638 tablet_replica.cc:333] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.068535  7638 raft_consensus.cc:2243] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.069180  7638 raft_consensus.cc:2272] T 739742728baa463f9d9f5e5884dd2e30 P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.071949  7638 tablet_replica.cc:333] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.072778  7638 raft_consensus.cc:2243] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.073385  7638 raft_consensus.cc:2272] T 7884996e8b84433f80124e36111f58ac P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.076277  7638 tablet_replica.cc:333] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.077082  7638 raft_consensus.cc:2243] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.077672  7638 raft_consensus.cc:2272] T 31c225a50048451db8964738c5db7540 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:23.078634  9414 consensus_peers.cc:597] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:23.080403  7638 tablet_replica.cc:333] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.081207  7638 raft_consensus.cc:2243] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.082393  7638 raft_consensus.cc:2272] T d4f573b86fda4b3e8fd982da9615dc8e P 112105225cc140459688be5d31dcb071 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.085214  7638 tablet_replica.cc:333] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.085943  7638 raft_consensus.cc:2243] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.086513  7638 raft_consensus.cc:2272] T 1f45eaa0e72045d5ae7ba8f95ca2ce75 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.089182  7638 tablet_replica.cc:333] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.090010  7638 raft_consensus.cc:2243] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.091313  7638 raft_consensus.cc:2272] T 4ac343934349411088cbf7a05932c2a5 P 112105225cc140459688be5d31dcb071 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:23.093767  9414 consensus_peers.cc:597] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:23.094003  7638 tablet_replica.cc:333] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.094826  7638 raft_consensus.cc:2243] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.095966  7638 raft_consensus.cc:2272] T 3006e1c3fb5b4c66852cbc955265f1cb P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.098547  7638 tablet_replica.cc:333] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.099270  7638 raft_consensus.cc:2243] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.099933  7638 raft_consensus.cc:2272] T 410ca1c9d2fd42c68bdf6bb996508fd9 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.102632  7638 tablet_replica.cc:333] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.103490  7638 raft_consensus.cc:2243] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.104591  7638 raft_consensus.cc:2272] T fd126387f2fb45f98d7ca0807566c688 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:23.105819  9414 consensus_peers.cc:597] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 -> Peer b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): Couldn't send request to peer b60591217ee24378bb66a86d6d0dda52. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:23.107436  7638 tablet_replica.cc:333] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.108268  7638 raft_consensus.cc:2243] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.109586  7638 raft_consensus.cc:2272] T 53864dd6ef544420915d557a63d5cd91 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.112293  7638 tablet_replica.cc:333] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.113039  7638 raft_consensus.cc:2243] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.114214  7638 raft_consensus.cc:2272] T a07b06e72d474091a0bcf7de181b509f P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.116779  7638 tablet_replica.cc:333] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.117574  7638 raft_consensus.cc:2243] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.118492  7638 raft_consensus.cc:2272] T 0192ad465a724ab581b9529345f73bba P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.121011  7638 tablet_replica.cc:333] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.121742  7638 raft_consensus.cc:2243] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.122727  7638 raft_consensus.cc:2272] T 3f192c6b771043a893d6fd420c9b9e7d P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.125270  7638 tablet_replica.cc:333] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.125981  7638 raft_consensus.cc:2243] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.126978  7638 raft_consensus.cc:2272] T 4bb4c86bcc704ac2add360ac3e5823df P 112105225cc140459688be5d31dcb071 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.129630  7638 tablet_replica.cc:333] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.130327  7638 raft_consensus.cc:2243] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.130897  7638 raft_consensus.cc:2272] T 33a0adf8736244d3b208cbc85700f0f9 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.133363  7638 tablet_replica.cc:333] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.134016  7638 raft_consensus.cc:2243] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.135016  7638 raft_consensus.cc:2272] T 029b57162d72422bba468c7d86b67fca P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.137446  7638 tablet_replica.cc:333] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.138139  7638 raft_consensus.cc:2243] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.139181  7638 raft_consensus.cc:2272] T dc6dcf2342014ed6b65dd5af29d18d3b P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.141743  7638 tablet_replica.cc:333] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.142470  7638 raft_consensus.cc:2243] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.143438  7638 raft_consensus.cc:2272] T 011b370821004185a003be210a525c01 P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.145872  7638 tablet_replica.cc:333] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.146550  7638 raft_consensus.cc:2243] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.147241  7638 raft_consensus.cc:2272] T 9402cffd3f284d28bf964bd56144a185 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.149768  7638 tablet_replica.cc:333] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.150468  7638 raft_consensus.cc:2243] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.151528  7638 raft_consensus.cc:2272] T 265e5f9784a6426482f5948fbfcc995c P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.154054  7638 tablet_replica.cc:333] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.154749  7638 raft_consensus.cc:2243] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.155623  7638 raft_consensus.cc:2272] T fe13736604454be3b391b8e946192961 P 112105225cc140459688be5d31dcb071 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.158191  7638 tablet_replica.cc:333] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.158851  7638 raft_consensus.cc:2243] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.159868  7638 raft_consensus.cc:2272] T 234e3fa2a66f42f3a5a755f5f4adc858 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.162108  7638 tablet_replica.cc:333] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.162662  7638 raft_consensus.cc:2243] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.163540  7638 raft_consensus.cc:2272] T 6fa55bf932444ccebb2b7f6583ead84f P 112105225cc140459688be5d31dcb071 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.165575  7638 tablet_replica.cc:333] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.166081  7638 raft_consensus.cc:2243] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.166889  7638 raft_consensus.cc:2272] T dada29d2380a453682f5dfef36dcf407 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.168721  7638 tablet_replica.cc:333] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.169200  7638 raft_consensus.cc:2243] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.169843  7638 raft_consensus.cc:2272] T 7bff3e7dddcd4dbebe38db1fa4eaae71 P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.171752  7638 tablet_replica.cc:333] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.172240  7638 raft_consensus.cc:2243] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.172690  7638 raft_consensus.cc:2272] T db96d769069547e38edb50dbebdbe00a P 112105225cc140459688be5d31dcb071 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.174460  7638 tablet_replica.cc:333] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.174978  7638 raft_consensus.cc:2243] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.175851  7638 raft_consensus.cc:2272] T 75401b4ca97a4c1eb75a016ff2124379 P 112105225cc140459688be5d31dcb071 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.177831  7638 tablet_replica.cc:333] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.178325  7638 raft_consensus.cc:2243] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.179159  7638 raft_consensus.cc:2272] T 1a0644aef66444b58e8633da6993535c P 112105225cc140459688be5d31dcb071 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.181000  7638 tablet_replica.cc:333] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.181547  7638 raft_consensus.cc:2243] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.182122  7638 raft_consensus.cc:2272] T 932d7bc525ba4af99f7d6b52fc1d7337 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.184016  7638 tablet_replica.cc:333] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.184473  7638 raft_consensus.cc:2243] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.185276  7638 raft_consensus.cc:2272] T c5bf6ec0e6a745d0967928a3361eb8d8 P 112105225cc140459688be5d31dcb071 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.187273  7638 tablet_replica.cc:333] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071: stopping tablet replica
I20260709 13:03:23.187760  7638 raft_consensus.cc:2243] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.188549  7638 raft_consensus.cc:2272] T fc45d3a2731b49daacb16548dc742815 P 112105225cc140459688be5d31dcb071 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.525331  7638 tablet_server.cc:196] TabletServer@127.7.117.132:0 shutdown complete.
I20260709 13:03:23.587527  7638 master.cc:562] Master@127.7.117.190:45111 shutting down...
W20260709 13:03:23.622401  8695 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:03:23.622702  8695 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:03:23.622910  8695 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:03:33.954813  8634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111) [suppressed 485 similar messages]
W20260709 13:03:33.958527  8634 catalog_manager.cc:4755] TS 29c7934ee528454785432732d01ed0a0 (127.7.117.130:44185): ChangeConfig:REMOVE_PEER RPC failed for tablet fe13736604454be3b391b8e946192961: Network error: Client connection negotiation failed: client connection to 127.7.117.130:44185: connect: Connection refused (error 111)
W20260709 13:03:35.996290  8633 catalog_manager.cc:4755] TS b60591217ee24378bb66a86d6d0dda52 (127.7.117.129:39991): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet d3e6b55229c842abb4b6a8551191af12: Network error: Client connection negotiation failed: client connection to 127.7.117.129:39991: connect: Connection refused (error 111)
I20260709 13:03:37.239933  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.240950  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.241376  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5b6e9771cda34fa68ec7172c32ecd31b: stopping tablet replica
I20260709 13:03:37.391454  7638 master.cc:584] Master@127.7.117.190:45111 shutdown complete.
I20260709 13:03:37.441988  7638 test_util.cc:182] -----------------------------------------------
I20260709 13:03:37.442204  7638 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935877741-7638-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (258632 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:03:37.445592  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:42727
I20260709 13:03:37.446650  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:37.451967 16825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.452458 16826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.454528 16828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.455049  7638 server_base.cc:1061] running on GCE node
I20260709 13:03:37.458087  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:37.458323  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:37.458477  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602217458461 us; error 0 us; skew 500 ppm
I20260709 13:03:37.459003  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:37.461479  7638 webserver.cc:533] Webserver started at http://127.7.117.190:38431/ using document root <none> and password file <none>
I20260709 13:03:37.461982  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:37.462186  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:37.462459  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:37.463737  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "1c7494c510344ab79b2f885aca8785d5"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.468137  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:03:37.471515 16833 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.472961  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:03:37.473248  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "1c7494c510344ab79b2f885aca8785d5"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.473618  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:37.485849  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:37.487134  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:37.526690  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:42727
I20260709 13:03:37.526808 16884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:42727 every 8 connection(s)
I20260709 13:03:37.530793 16885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:37.545251 16885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5: Bootstrap starting.
I20260709 13:03:37.550204 16885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.554512 16885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5: No bootstrap required, opened a new log
I20260709 13:03:37.556846 16885 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER }
I20260709 13:03:37.557431 16885 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.557650 16885 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7494c510344ab79b2f885aca8785d5, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.558264 16885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER }
I20260709 13:03:37.558745 16885 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:37.558960 16885 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:37.559283 16885 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.563900 16885 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER }
I20260709 13:03:37.564424 16885 leader_election.cc:304] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c7494c510344ab79b2f885aca8785d5; no voters: 
I20260709 13:03:37.565630 16885 leader_election.cc:290] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:37.566022 16888 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.567338 16888 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 1 LEADER]: Becoming Leader. State: Replica: 1c7494c510344ab79b2f885aca8785d5, State: Running, Role: LEADER
I20260709 13:03:37.568025 16888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER }
I20260709 13:03:37.568578 16885 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:37.574796 16890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7494c510344ab79b2f885aca8785d5. Latest consensus state: current_term: 1 leader_uuid: "1c7494c510344ab79b2f885aca8785d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER } }
I20260709 13:03:37.575054 16889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c7494c510344ab79b2f885aca8785d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7494c510344ab79b2f885aca8785d5" member_type: VOTER } }
I20260709 13:03:37.575519 16890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:37.575562 16889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:37.577809 16897 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:37.585474 16897 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:37.586444  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:37.594465 16897 catalog_manager.cc:1383] Generated new cluster ID: 0531b04fbc4446f7979a15f905ebfdc4
I20260709 13:03:37.594765 16897 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:37.617004 16897 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:37.618389 16897 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:37.635110 16897 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5: Generated new TSK 0
I20260709 13:03:37.635762 16897 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:37.652736  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:37.659471 16906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.660127 16907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.663874 16909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.663931  7638 server_base.cc:1061] running on GCE node
I20260709 13:03:37.667865  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:37.668090  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:37.668215  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602217668203 us; error 0 us; skew 500 ppm
I20260709 13:03:37.668692  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:37.671155  7638 webserver.cc:533] Webserver started at http://127.7.117.129:33137/ using document root <none> and password file <none>
I20260709 13:03:37.671612  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:37.671769  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:37.671988  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:37.673106  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "64734f14219348f59d1cf8700416565c"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.677744  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:03:37.681039 16914 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.682178  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:03:37.682555  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "64734f14219348f59d1cf8700416565c"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.682866  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:37.704238  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:37.705565  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:37.707067  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:37.709488  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:37.709728  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.709976  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:37.710166  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.751873  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:38761
I20260709 13:03:37.751969 16976 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:38761 every 8 connection(s)
I20260709 13:03:37.757799  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:37.765230 16981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.767060 16982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.769510 16984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.770406  7638 server_base.cc:1061] running on GCE node
I20260709 13:03:37.772672  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:37.772943  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:37.773154  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602217773130 us; error 0 us; skew 500 ppm
I20260709 13:03:37.773936  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:37.775972 16977 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42727
I20260709 13:03:37.776372 16977 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:37.777022  7638 webserver.cc:533] Webserver started at http://127.7.117.130:43547/ using document root <none> and password file <none>
I20260709 13:03:37.777441 16977 heartbeater.cc:507] Master 127.7.117.190:42727 requested a full tablet report, sending...
I20260709 13:03:37.777769  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:37.778045  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:37.778362  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:37.779896  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.780001 16850 ts_manager.cc:194] Registered new tserver with Master: 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:37.781975 16850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35474
I20260709 13:03:37.785152  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:03:37.788985 16989 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.789870  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:37.790181  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.790477  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:37.808491  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:37.809798  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:37.811234  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:37.813459  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:37.813694  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.813930  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:37.814100  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.855571  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:42149
I20260709 13:03:37.855683 17051 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:42149 every 8 connection(s)
I20260709 13:03:37.860571  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:37.867767 17055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.869555 17056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.871765  7638 server_base.cc:1061] running on GCE node
W20260709 13:03:37.872099 17058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.872983 17052 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42727
I20260709 13:03:37.873271  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:03:37.873387 17052 heartbeater.cc:461] Registering TS with master...
W20260709 13:03:37.873669  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:37.873893  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602217873878 us; error 0 us; skew 500 ppm
I20260709 13:03:37.874353 17052 heartbeater.cc:507] Master 127.7.117.190:42727 requested a full tablet report, sending...
I20260709 13:03:37.874507  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:37.876542 16850 ts_manager.cc:194] Registered new tserver with Master: ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:37.877475  7638 webserver.cc:533] Webserver started at http://127.7.117.131:42403/ using document root <none> and password file <none>
I20260709 13:03:37.878011  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:37.877986 16850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35482
I20260709 13:03:37.878541  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:37.878875  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:37.880318  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "1cc950e241a640e0912a73ed9309b1e5"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.884943  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:03:37.888422 17063 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.889196  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:03:37.889570  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "1cc950e241a640e0912a73ed9309b1e5"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.889838  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:37.906898  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:37.908164  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:37.909699  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:37.911844  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:37.912083  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.912326  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:37.912509  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.953413  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:43115
I20260709 13:03:37.953527 17125 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:43115 every 8 connection(s)
I20260709 13:03:37.958470  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:37.966332 17129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:37.967535 17130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.969687  7638 server_base.cc:1061] running on GCE node
W20260709 13:03:37.971237 17132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:37.972157 17126 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42727
I20260709 13:03:37.972437  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:03:37.972551 17126 heartbeater.cc:461] Registering TS with master...
W20260709 13:03:37.972790  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:37.973028  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602217973012 us; error 0 us; skew 500 ppm
I20260709 13:03:37.973557 17126 heartbeater.cc:507] Master 127.7.117.190:42727 requested a full tablet report, sending...
I20260709 13:03:37.973769  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:37.975916 16850 ts_manager.cc:194] Registered new tserver with Master: 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:37.976516  7638 webserver.cc:533] Webserver started at http://127.7.117.132:35189/ using document root <none> and password file <none>
I20260709 13:03:37.977104  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:37.977389  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:37.977542 16850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35494
I20260709 13:03:37.977665  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:37.979053  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-3-root/instance:
uuid: "8267993b082f449891e464bc4dee65f7"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.983530  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:03:37.987098 17137 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:37.987857  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:03:37.988164  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
uuid: "8267993b082f449891e464bc4dee65f7"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:37.988467  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:38.008147  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:38.009538  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:38.011042  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:38.013332  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:38.013568  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:38.013805  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:38.013975  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:38.054930  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.132:38527
I20260709 13:03:38.055025 17199 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.132:38527 every 8 connection(s)
I20260709 13:03:38.069180 17200 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42727
I20260709 13:03:38.069629 17200 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:38.070488 17200 heartbeater.cc:507] Master 127.7.117.190:42727 requested a full tablet report, sending...
I20260709 13:03:38.072707 16850 ts_manager.cc:194] Registered new tserver with Master: 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:38.072808  7638 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01473303s
I20260709 13:03:38.074193 16850 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35506
I20260709 13:03:38.099146 16850 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35520:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:03:38.343770 17091 tablet_service.cc:1511] Processing CreateTablet for tablet 61ae0ddb123d4d8bb5b071178e546931 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:38.345631 17091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61ae0ddb123d4d8bb5b071178e546931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.348244 17165 tablet_service.cc:1511] Processing CreateTablet for tablet 61ae0ddb123d4d8bb5b071178e546931 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:38.349097 17090 tablet_service.cc:1511] Processing CreateTablet for tablet 355dd8ce521a412c8b28a5988e30bf7e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:38.349753 17165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61ae0ddb123d4d8bb5b071178e546931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.350517 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355dd8ce521a412c8b28a5988e30bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.354530 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 61ae0ddb123d4d8bb5b071178e546931 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:38.354640 17089 tablet_service.cc:1511] Processing CreateTablet for tablet afb0f82169da45819893f31678f63f76 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:38.356003 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61ae0ddb123d4d8bb5b071178e546931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.356197 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb0f82169da45819893f31678f63f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.358515 17088 tablet_service.cc:1511] Processing CreateTablet for tablet f0c2977ecbd943b3bbffc28c746df99f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:38.359120 17164 tablet_service.cc:1511] Processing CreateTablet for tablet de112b6726e84fdca382fa7b9a903c49 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:38.360031 17088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c2977ecbd943b3bbffc28c746df99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.359853 16940 tablet_service.cc:1511] Processing CreateTablet for tablet de112b6726e84fdca382fa7b9a903c49 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:38.360498 17164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de112b6726e84fdca382fa7b9a903c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.362463 16939 tablet_service.cc:1511] Processing CreateTablet for tablet f0c2977ecbd943b3bbffc28c746df99f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:38.367234 16938 tablet_service.cc:1511] Processing CreateTablet for tablet 2abd5bac63004970887f5d75e88e9a0f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:38.370536 16936 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed0bc1914444d179b36b1253f15c65c (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:38.372910 16935 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed34ea43214b00a76e89f8b09249de (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:38.374192 17163 tablet_service.cc:1511] Processing CreateTablet for tablet afb0f82169da45819893f31678f63f76 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:38.375581 17163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb0f82169da45819893f31678f63f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.375696 16934 tablet_service.cc:1511] Processing CreateTablet for tablet bf62f622f0524a91bf77aab05b4c9d3f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:38.377243 16937 tablet_service.cc:1511] Processing CreateTablet for tablet d4035d55f3884082a4ba26e608bfdd6b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:38.361229 16940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de112b6726e84fdca382fa7b9a903c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.384861 16936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed0bc1914444d179b36b1253f15c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.386921 17162 tablet_service.cc:1511] Processing CreateTablet for tablet 2abd5bac63004970887f5d75e88e9a0f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:38.388334 17162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abd5bac63004970887f5d75e88e9a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.389657 16934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf62f622f0524a91bf77aab05b4c9d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.391880 16938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abd5bac63004970887f5d75e88e9a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.397845 16939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c2977ecbd943b3bbffc28c746df99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.399448 17087 tablet_service.cc:1511] Processing CreateTablet for tablet d4035d55f3884082a4ba26e608bfdd6b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:38.400840 17087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4035d55f3884082a4ba26e608bfdd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.401630 16937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4035d55f3884082a4ba26e608bfdd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.358462 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 355dd8ce521a412c8b28a5988e30bf7e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:38.404352 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355dd8ce521a412c8b28a5988e30bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.405743 16935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed34ea43214b00a76e89f8b09249de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.409668 17161 tablet_service.cc:1511] Processing CreateTablet for tablet d4035d55f3884082a4ba26e608bfdd6b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:38.411478 17161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4035d55f3884082a4ba26e608bfdd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.423223 17086 tablet_service.cc:1511] Processing CreateTablet for tablet 3e944c9ade604302b16ffd9d285523a4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:38.424798 17086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e944c9ade604302b16ffd9d285523a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.435400 16933 tablet_service.cc:1511] Processing CreateTablet for tablet accaf859802f485d9a8e81e692ccdb50 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:38.436931 16933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accaf859802f485d9a8e81e692ccdb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.468132 17160 tablet_service.cc:1511] Processing CreateTablet for tablet 3e944c9ade604302b16ffd9d285523a4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:38.470355 17160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e944c9ade604302b16ffd9d285523a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.523684 17159 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed34ea43214b00a76e89f8b09249de (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:38.525079 17159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed34ea43214b00a76e89f8b09249de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.527508 17090 tablet_service.cc:1511] Processing CreateTablet for tablet bf62f622f0524a91bf77aab05b4c9d3f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:38.532910 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf62f622f0524a91bf77aab05b4c9d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.527845 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:38.527763 17158 tablet_service.cc:1511] Processing CreateTablet for tablet bf62f622f0524a91bf77aab05b4c9d3f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:38.546159 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.546303 17158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf62f622f0524a91bf77aab05b4c9d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.529040 17089 tablet_service.cc:1511] Processing CreateTablet for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:38.550117 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.551445 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1ffde0abe4f029d91b01b8d08d2b7 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:38.530596 17091 tablet_service.cc:1511] Processing CreateTablet for tablet accaf859802f485d9a8e81e692ccdb50 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:38.529091 17085 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1ffde0abe4f029d91b01b8d08d2b7 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:38.531424 17156 tablet_service.cc:1511] Processing CreateTablet for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:38.529701 17157 tablet_service.cc:1511] Processing CreateTablet for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:38.552286 17084 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8c7883a65740a5842d54e35490a6e1 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:38.554441 16940 tablet_service.cc:1511] Processing CreateTablet for tablet 006cb6681bb949508a3ac253c5a64661 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:38.566047 17156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.566249 17220 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:38.567075 17161 tablet_service.cc:1511] Processing CreateTablet for tablet a8756dd718c74c838e90911ac1fb65e5 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:38.568068 17091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accaf859802f485d9a8e81e692ccdb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.560477 17083 tablet_service.cc:1511] Processing CreateTablet for tablet 006cb6681bb949508a3ac253c5a64661 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:38.571722 17085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1ffde0abe4f029d91b01b8d08d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.572422 17083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 006cb6681bb949508a3ac253c5a64661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.572890 17157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.573387 17084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8c7883a65740a5842d54e35490a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.578084 17086 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed0bc1914444d179b36b1253f15c65c (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:38.579530 17086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed0bc1914444d179b36b1253f15c65c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:38.580065 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:38.559865 16937 tablet_service.cc:1511] Processing CreateTablet for tablet 16ada941539640f58a0f2ba47d6c1954 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:38.560086 17165 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8c7883a65740a5842d54e35490a6e1 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:38.596859 16935 tablet_service.cc:1511] Processing CreateTablet for tablet 7d970e7113e5440181d65961915967ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:38.601701 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1ffde0abe4f029d91b01b8d08d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.560933 16936 tablet_service.cc:1511] Processing CreateTablet for tablet a8756dd718c74c838e90911ac1fb65e5 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:38.606087 17082 tablet_service.cc:1511] Processing CreateTablet for tablet bd2354d787dc43f58fd361f6b4832507 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:38.604172 17083 tablet_service.cc:1511] Processing CreateTablet for tablet 16ada941539640f58a0f2ba47d6c1954 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:38.610340 17088 tablet_service.cc:1511] Processing CreateTablet for tablet 7d970e7113e5440181d65961915967ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:38.610594 17163 tablet_service.cc:1511] Processing CreateTablet for tablet 16ada941539640f58a0f2ba47d6c1954 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:38.611624 17082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2354d787dc43f58fd361f6b4832507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.614111 17219 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:38.614203 17218 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:38.618330 17090 tablet_service.cc:1511] Processing CreateTablet for tablet c71c71675c2b420aaaad7cd24eef4084 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:38.619692 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71c71675c2b420aaaad7cd24eef4084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.621120 17083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16ada941539640f58a0f2ba47d6c1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.622473 17160 tablet_service.cc:1511] Processing CreateTablet for tablet c71c71675c2b420aaaad7cd24eef4084 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:38.630076 17088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d970e7113e5440181d65961915967ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.635190 17162 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1ffde0abe4f029d91b01b8d08d2b7 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:38.640861 17220 tablet_bootstrap.cc:654] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.646186 16940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 006cb6681bb949508a3ac253c5a64661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.650094 16936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8756dd718c74c838e90911ac1fb65e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.651183 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 7eae2b6552a54c4bbaff1296163ac17b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:38.652487 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eae2b6552a54c4bbaff1296163ac17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.660418 16935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d970e7113e5440181d65961915967ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.660080 17161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8756dd718c74c838e90911ac1fb65e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.661851 16933 tablet_service.cc:1511] Processing CreateTablet for tablet 1ae68a5d259b476daae7a5665770a0cd (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:38.668716 17164 tablet_service.cc:1511] Processing CreateTablet for tablet 7eae2b6552a54c4bbaff1296163ac17b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:38.692420 17164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eae2b6552a54c4bbaff1296163ac17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.693836 17218 tablet_bootstrap.cc:654] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.693748 17087 tablet_service.cc:1511] Processing CreateTablet for tablet 1ae68a5d259b476daae7a5665770a0cd (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:38.695181 17087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae68a5d259b476daae7a5665770a0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.690834 17084 tablet_service.cc:1511] Processing CreateTablet for tablet bafbaa58b13148c2b916fe7ee7e234e6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:38.698387 17084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bafbaa58b13148c2b916fe7ee7e234e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.674592 17219 tablet_bootstrap.cc:654] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.698755 16934 tablet_service.cc:1511] Processing CreateTablet for tablet ed56f28edb0c443ead4f92414e0a29fa (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:38.657198 16938 tablet_service.cc:1511] Processing CreateTablet for tablet bd2354d787dc43f58fd361f6b4832507 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:38.701227 17091 tablet_service.cc:1511] Processing CreateTablet for tablet ed56f28edb0c443ead4f92414e0a29fa (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:38.702492 17091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed56f28edb0c443ead4f92414e0a29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.702915 17082 tablet_service.cc:1511] Processing CreateTablet for tablet d3c68d18d16941a9b3197081006477dc (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:38.704161 17082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c68d18d16941a9b3197081006477dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.705843 17162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1ffde0abe4f029d91b01b8d08d2b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.706929 17089 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8e87277d714ded843585986eae389e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:38.708461 17165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8c7883a65740a5842d54e35490a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.682809 16942 tablet_service.cc:1511] Processing CreateTablet for tablet cdaa7a451fbb426d868eef2a6abd8849 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:38.674425 17159 tablet_service.cc:1511] Processing CreateTablet for tablet bd2354d787dc43f58fd361f6b4832507 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:38.712365 17159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2354d787dc43f58fd361f6b4832507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.713788 17220 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:38.714326 17220 ts_tablet_manager.cc:1403] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.148s	user 0.013s	sys 0.000s
I20260709 13:03:38.714277 16939 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8e87277d714ded843585986eae389e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:38.715926 17163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16ada941539640f58a0f2ba47d6c1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.721629 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8e87277d714ded843585986eae389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.725741 17156 tablet_service.cc:1511] Processing CreateTablet for tablet cdaa7a451fbb426d868eef2a6abd8849 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:38.727100 16937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16ada941539640f58a0f2ba47d6c1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.727859 16933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae68a5d259b476daae7a5665770a0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.731637 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdaa7a451fbb426d868eef2a6abd8849. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.736585 16938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2354d787dc43f58fd361f6b4832507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.738438 17083 tablet_service.cc:1511] Processing CreateTablet for tablet 855f8184b9834eeb8c9957f6a2df61f4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:38.739830 17083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 855f8184b9834eeb8c9957f6a2df61f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.745594 17088 tablet_service.cc:1511] Processing CreateTablet for tablet 4633fd2f819d45d893ebcdb54511d834 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:38.746129 17158 tablet_service.cc:1511] Processing CreateTablet for tablet bafbaa58b13148c2b916fe7ee7e234e6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:38.745594 17085 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed6b9a59b31479aaa96d38781de77d6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:38.727114 17156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdaa7a451fbb426d868eef2a6abd8849. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.753382 17160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71c71675c2b420aaaad7cd24eef4084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.753588 16940 tablet_service.cc:1511] Processing CreateTablet for tablet a9a2edef1b2a42bca289c34851377b55 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:38.754788 16939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8e87277d714ded843585986eae389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.758630 16934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed56f28edb0c443ead4f92414e0a29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.760048 17158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bafbaa58b13148c2b916fe7ee7e234e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.777611 17088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633fd2f819d45d893ebcdb54511d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.779443 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec250363e734059ae59f362df78c792 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:38.783694 17086 tablet_service.cc:1511] Processing CreateTablet for tablet 162f7d6234e549cc968efac4ee8846ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:38.785058 16977 heartbeater.cc:499] Master 127.7.117.190:42727 was elected leader, sending a full tablet report...
I20260709 13:03:38.788378 17090 tablet_service.cc:1511] Processing CreateTablet for tablet a40bdee1b5c3486b94412af27867b738 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:38.785218 16940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a2edef1b2a42bca289c34851377b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.785089 17086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162f7d6234e549cc968efac4ee8846ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.765269 17084 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec250363e734059ae59f362df78c792 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:38.811025 17218 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:38.811559 17218 ts_tablet_manager.cc:1403] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.220s	user 0.003s	sys 0.010s
I20260709 13:03:38.809245 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec250363e734059ae59f362df78c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.769129 17157 tablet_service.cc:1511] Processing CreateTablet for tablet ed56f28edb0c443ead4f92414e0a29fa (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:38.817205 17165 tablet_service.cc:1511] Processing CreateTablet for tablet d3c68d18d16941a9b3197081006477dc (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:38.818740 17220 raft_consensus.cc:359] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:38.827410 17220 raft_consensus.cc:385] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.827708 17220 raft_consensus.cc:740] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.827903 17218 raft_consensus.cc:359] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:38.828747 17218 raft_consensus.cc:385] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.828599 17220 consensus_queue.cc:260] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:38.829131 17218 raft_consensus.cc:740] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.824909 17157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed56f28edb0c443ead4f92414e0a29fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.831157 16935 tablet_service.cc:1511] Processing CreateTablet for tablet a40bdee1b5c3486b94412af27867b738 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:38.832458 16935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a40bdee1b5c3486b94412af27867b738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.833446 17085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed6b9a59b31479aaa96d38781de77d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.836326 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 162f7d6234e549cc968efac4ee8846ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:38.839538 17084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec250363e734059ae59f362df78c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.768671 16936 tablet_service.cc:1511] Processing CreateTablet for tablet 855f8184b9834eeb8c9957f6a2df61f4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:38.845466 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a40bdee1b5c3486b94412af27867b738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.828967 16938 tablet_service.cc:1511] Processing CreateTablet for tablet afe5ed1a667143e0869e8f06a2a27686 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:38.848673 17165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c68d18d16941a9b3197081006477dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.859613 17087 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd44ab20c334b99964d344244e650ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:38.860980 17087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd44ab20c334b99964d344244e650ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.866011 17220 ts_tablet_manager.cc:1434] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.151s	user 0.011s	sys 0.008s
I20260709 13:03:38.866434 17083 tablet_service.cc:1511] Processing CreateTablet for tablet 2db96a4ed139460aa021acdd63b9b529 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:38.866839 16933 tablet_service.cc:1511] Processing CreateTablet for tablet c6ce161acdcd435a8f869ae50a6ea162 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:38.867722 17083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2db96a4ed139460aa021acdd63b9b529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.867574 17088 tablet_service.cc:1511] Processing CreateTablet for tablet 117045952bb14176abd739933fa51259 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:38.868242 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162f7d6234e549cc968efac4ee8846ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.868794 17088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117045952bb14176abd739933fa51259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.846293 17218 consensus_queue.cc:260] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:38.870478 16938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe5ed1a667143e0869e8f06a2a27686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.870638 17200 heartbeater.cc:499] Master 127.7.117.190:42727 was elected leader, sending a full tablet report...
I20260709 13:03:38.879858 16933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ce161acdcd435a8f869ae50a6ea162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.881031 17052 heartbeater.cc:499] Master 127.7.117.190:42727 was elected leader, sending a full tablet report...
I20260709 13:03:38.885671 17162 tablet_service.cc:1511] Processing CreateTablet for tablet a9a2edef1b2a42bca289c34851377b55 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:38.885958 17164 tablet_service.cc:1511] Processing CreateTablet for tablet 855f8184b9834eeb8c9957f6a2df61f4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:38.886703 16936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 855f8184b9834eeb8c9957f6a2df61f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.886490 17158 tablet_service.cc:1511] Processing CreateTablet for tablet 4633fd2f819d45d893ebcdb54511d834 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:38.887023 17162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a2edef1b2a42bca289c34851377b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.887763 17158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633fd2f819d45d893ebcdb54511d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.888288 17220 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:38.908931 17224 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.909586 17224 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:38.912293 17224 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:38.918552 16939 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd44ab20c334b99964d344244e650ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:38.919869 16939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd44ab20c334b99964d344244e650ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.915222 17082 tablet_service.cc:1511] Processing CreateTablet for tablet a97fcf0cdbc04350a9368ac4f9972f72 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:38.922492 17082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fcf0cdbc04350a9368ac4f9972f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.924669 17091 tablet_service.cc:1511] Processing CreateTablet for tablet 0395376e4b3d44a0a28a7bb199a21307 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:38.926293 17091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0395376e4b3d44a0a28a7bb199a21307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.930625 17160 tablet_service.cc:1511] Processing CreateTablet for tablet afe5ed1a667143e0869e8f06a2a27686 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:38.932298 17160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe5ed1a667143e0869e8f06a2a27686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:38.962301 17218 ts_tablet_manager.cc:1434] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.150s	user 0.014s	sys 0.014s
I20260709 13:03:38.963136 17126 heartbeater.cc:499] Master 127.7.117.190:42727 was elected leader, sending a full tablet report...
I20260709 13:03:38.965899 17218 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:38.968647 17162 tablet_service.cc:1511] Processing CreateTablet for tablet 162f7d6234e549cc968efac4ee8846ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:38.969642 17219 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:38.981511 17219 ts_tablet_manager.cc:1403] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.368s	user 0.012s	sys 0.009s
I20260709 13:03:38.983325 17218 tablet_bootstrap.cc:654] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.904975 17163 tablet_service.cc:1511] Processing CreateTablet for tablet c6ce161acdcd435a8f869ae50a6ea162 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:38.996079 17220 tablet_bootstrap.cc:654] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.008302 17161 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed6b9a59b31479aaa96d38781de77d6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:38.998714 17164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 855f8184b9834eeb8c9957f6a2df61f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.019024 17156 tablet_service.cc:1511] Processing CreateTablet for tablet 2db96a4ed139460aa021acdd63b9b529 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:39.027606 17089 tablet_service.cc:1511] Processing CreateTablet for tablet edd447a4120b4d378b710f3f18d0541f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:39.050249 17158 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd44ab20c334b99964d344244e650ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:39.050920 16937 tablet_service.cc:1511] Processing CreateTablet for tablet 117045952bb14176abd739933fa51259 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:39.052215 16937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117045952bb14176abd739933fa51259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.053141 17017 tablet_service.cc:1511] Processing CreateTablet for tablet 355dd8ce521a412c8b28a5988e30bf7e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:39.054467 17017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355dd8ce521a412c8b28a5988e30bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.029600 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd447a4120b4d378b710f3f18d0541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.056761 17085 tablet_service.cc:1511] Processing CreateTablet for tablet 0eccff1f699a4e4387998ef0842e7d33 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:03:39.061757 17201 tablet.cc:2406] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:39.064599 17016 tablet_service.cc:1511] Processing CreateTablet for tablet de112b6726e84fdca382fa7b9a903c49 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:39.067075 17016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de112b6726e84fdca382fa7b9a903c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.069423 17161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed6b9a59b31479aaa96d38781de77d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.064610 17015 tablet_service.cc:1511] Processing CreateTablet for tablet afb0f82169da45819893f31678f63f76 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:39.070745 17015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb0f82169da45819893f31678f63f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.078980 17014 tablet_service.cc:1511] Processing CreateTablet for tablet f0c2977ecbd943b3bbffc28c746df99f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:39.080412 17014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c2977ecbd943b3bbffc28c746df99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.081156 17008 tablet_service.cc:1511] Processing CreateTablet for tablet accaf859802f485d9a8e81e692ccdb50 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:39.082549 17008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accaf859802f485d9a8e81e692ccdb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.082418 17013 tablet_service.cc:1511] Processing CreateTablet for tablet 2abd5bac63004970887f5d75e88e9a0f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:39.083840 17013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abd5bac63004970887f5d75e88e9a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.086505 17012 tablet_service.cc:1511] Processing CreateTablet for tablet 3e944c9ade604302b16ffd9d285523a4 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:39.087843 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e944c9ade604302b16ffd9d285523a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.090492 17011 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed0bc1914444d179b36b1253f15c65c (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:39.091883 17011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed0bc1914444d179b36b1253f15c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.094485 17010 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed34ea43214b00a76e89f8b09249de (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:39.095857 17010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed34ea43214b00a76e89f8b09249de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.099318 17225 raft_consensus.cc:493] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.099529 16939 tablet_service.cc:1511] Processing CreateTablet for tablet 0395376e4b3d44a0a28a7bb199a21307 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:39.108150 16934 tablet_service.cc:1511] Processing CreateTablet for tablet 100511a7ba6744e08fb2dec95a371354 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:39.098453 17009 tablet_service.cc:1511] Processing CreateTablet for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:39.110021 17219 raft_consensus.cc:359] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.110440 17225 raft_consensus.cc:515] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:39.103914 17156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2db96a4ed139460aa021acdd63b9b529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.111802 16939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0395376e4b3d44a0a28a7bb199a21307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.110850 17219 raft_consensus.cc:385] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.063414 17085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eccff1f699a4e4387998ef0842e7d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.110082 17009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b5b7c58e054ac6ac9e77e29ed86e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.126040 17163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ce161acdcd435a8f869ae50a6ea162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.116796 17219 raft_consensus.cc:740] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.137010 17162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162f7d6234e549cc968efac4ee8846ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.138541 17225 leader_election.cc:290] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:39.065692 16936 tablet_service.cc:1511] Processing CreateTablet for tablet a97fcf0cdbc04350a9368ac4f9972f72 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:39.152381 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:39.155077 17141 leader_election.cc:343] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.159704 16938 tablet_service.cc:1511] Processing CreateTablet for tablet 0eccff1f699a4e4387998ef0842e7d33 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:39.161345 16934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100511a7ba6744e08fb2dec95a371354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.163256 17219 consensus_queue.cc:260] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.170845 17157 tablet_service.cc:1511] Processing CreateTablet for tablet 6a02330036da4716bc0da7d166bdea11 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:39.173183 16941 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2819a664db491aa04d5c6ec4c13708 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:39.175776 16942 tablet_service.cc:1511] Processing CreateTablet for tablet 6a02330036da4716bc0da7d166bdea11 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:39.182418 17159 tablet_service.cc:1511] Processing CreateTablet for tablet e797287dc34d42e3838f8f5a3ba68985 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:39.188047 16935 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca37e8a20c2447983a81703b8706d51 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:39.193534 17158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd44ab20c334b99964d344244e650ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.197593 16936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fcf0cdbc04350a9368ac4f9972f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.197937 16940 tablet_service.cc:1511] Processing CreateTablet for tablet a12c1a9abfa94488a9aefb37cb9ecdf0 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:39.200856 17165 tablet_service.cc:1511] Processing CreateTablet for tablet 100511a7ba6744e08fb2dec95a371354 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:39.250907 17165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100511a7ba6744e08fb2dec95a371354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.253724 17159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e797287dc34d42e3838f8f5a3ba68985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.254374 17084 tablet_service.cc:1511] Processing CreateTablet for tablet e797287dc34d42e3838f8f5a3ba68985 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:39.255770 17084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e797287dc34d42e3838f8f5a3ba68985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.257840 16935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca37e8a20c2447983a81703b8706d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.258812 17090 tablet_service.cc:1511] Processing CreateTablet for tablet a12c1a9abfa94488a9aefb37cb9ecdf0 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:39.260061 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12c1a9abfa94488a9aefb37cb9ecdf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.283776 17218 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:39.284380 17218 ts_tablet_manager.cc:1403] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.319s	user 0.018s	sys 0.004s
I20260709 13:03:39.286545 16938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eccff1f699a4e4387998ef0842e7d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.289184 16942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a02330036da4716bc0da7d166bdea11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.296500 16941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2819a664db491aa04d5c6ec4c13708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.297461 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:39.299324 17140 leader_election.cc:343] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.299789 17140 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:39.300729 17226 raft_consensus.cc:2749] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.325479 17157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a02330036da4716bc0da7d166bdea11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.335033 17017 tablet_service.cc:1511] Processing CreateTablet for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:39.335943 17089 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca37e8a20c2447983a81703b8706d51 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:39.336386 17017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33c172d9d8e34d8b86a2d40dd97cfc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.332193 16940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12c1a9abfa94488a9aefb37cb9ecdf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.337221 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca37e8a20c2447983a81703b8706d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.343905 16936 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17e18d23d343e6894f11dc02b2eb75 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:39.346335 16933 tablet_service.cc:1511] Processing CreateTablet for tablet e5e1d0f0dc3240608259e975a8ea33ec (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:39.346557 17012 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8c7883a65740a5842d54e35490a6e1 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:39.347174 17010 tablet_service.cc:1511] Processing CreateTablet for tablet 006cb6681bb949508a3ac253c5a64661 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:39.350374 17219 ts_tablet_manager.cc:1434] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.368s	user 0.015s	sys 0.046s
I20260709 13:03:39.351441 17219 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:39.351899 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8c7883a65740a5842d54e35490a6e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.352560 17010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 006cb6681bb949508a3ac253c5a64661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.356666 17015 tablet_service.cc:1511] Processing CreateTablet for tablet a8756dd718c74c838e90911ac1fb65e5 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:39.400736 17015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8756dd718c74c838e90911ac1fb65e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.402256 17088 tablet_service.cc:1511] Processing CreateTablet for tablet a9bdaab91b264f3ea1e21283b3c988ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:39.403544 17088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bdaab91b264f3ea1e21283b3c988ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.408298 17085 tablet_service.cc:1511] Processing CreateTablet for tablet 391757a5f74549289d0b597a84df6740 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:39.409664 17085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391757a5f74549289d0b597a84df6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.412315 17163 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2819a664db491aa04d5c6ec4c13708 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:39.413707 17163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2819a664db491aa04d5c6ec4c13708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.379777 17014 tablet_service.cc:1511] Processing CreateTablet for tablet c71c71675c2b420aaaad7cd24eef4084 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:39.381161 17016 tablet_service.cc:1511] Processing CreateTablet for tablet 7d970e7113e5440181d65961915967ad (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:39.420799 17016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d970e7113e5440181d65961915967ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.423308 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:39.383729 17013 tablet_service.cc:1511] Processing CreateTablet for tablet 7eae2b6552a54c4bbaff1296163ac17b (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:39.426020 17014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71c71675c2b420aaaad7cd24eef4084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.426610 17013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eae2b6552a54c4bbaff1296163ac17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.391505 17220 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:39.429708 17220 ts_tablet_manager.cc:1403] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.542s	user 0.030s	sys 0.009s
I20260709 13:03:39.430351 17162 tablet_service.cc:1511] Processing CreateTablet for tablet edd447a4120b4d378b710f3f18d0541f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:39.301054 17164 tablet_service.cc:1511] Processing CreateTablet for tablet a97fcf0cdbc04350a9368ac4f9972f72 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:39.395725 17156 tablet_service.cc:1511] Processing CreateTablet for tablet 0eccff1f699a4e4387998ef0842e7d33 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:03:39.424993 17066 leader_election.cc:343] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.362926 16933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e1d0f0dc3240608259e975a8ea33ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.414387 17082 tablet_service.cc:1511] Processing CreateTablet for tablet eb98a7c474ea41c3a428ce29909fc94f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:39.433393 17082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb98a7c474ea41c3a428ce29909fc94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.414438 17083 tablet_service.cc:1511] Processing CreateTablet for tablet 60c79060c3294f4fbe036c1fe48ae9bf (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:39.370841 17218 raft_consensus.cc:359] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.435288 17009 tablet_service.cc:1511] Processing CreateTablet for tablet bafbaa58b13148c2b916fe7ee7e234e6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:39.436117 17218 raft_consensus.cc:385] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.436486 17218 raft_consensus.cc:740] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.436628 17083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c79060c3294f4fbe036c1fe48ae9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.436966 17009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bafbaa58b13148c2b916fe7ee7e234e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.439152 16937 tablet_service.cc:1511] Processing CreateTablet for tablet 60c79060c3294f4fbe036c1fe48ae9bf (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:39.440743 16937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c79060c3294f4fbe036c1fe48ae9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.444877 16936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17e18d23d343e6894f11dc02b2eb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.445201 17090 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17e18d23d343e6894f11dc02b2eb75 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:39.445727 17237 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.446331 17237 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.455255 17161 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca37e8a20c2447983a81703b8706d51 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:39.446904 17090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17e18d23d343e6894f11dc02b2eb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.466687 17238 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:39.467954 17011 tablet_service.cc:1511] Processing CreateTablet for tablet 1ae68a5d259b476daae7a5665770a0cd (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:39.469272 17011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae68a5d259b476daae7a5665770a0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.478799 17008 tablet_service.cc:1511] Processing CreateTablet for tablet cdaa7a451fbb426d868eef2a6abd8849 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:39.480695 17164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fcf0cdbc04350a9368ac4f9972f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.476869 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:39.526656 17067 leader_election.cc:343] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.527169 17067 leader_election.cc:304] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:39.527369 16934 tablet_service.cc:1511] Processing CreateTablet for tablet eb98a7c474ea41c3a428ce29909fc94f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:39.528396 17225 raft_consensus.cc:2749] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.528673 16934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb98a7c474ea41c3a428ce29909fc94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.492306 17238 tablet_bootstrap.cc:654] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.493868 17220 raft_consensus.cc:359] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.532096 17220 raft_consensus.cc:385] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.532433 17220 raft_consensus.cc:740] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.463903 17218 consensus_queue.cc:260] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.511878 17159 tablet_service.cc:1511] Processing CreateTablet for tablet e5e1d0f0dc3240608259e975a8ea33ec (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:39.533993 17159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e1d0f0dc3240608259e975a8ea33ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.541909 17218 ts_tablet_manager.cc:1434] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.257s	user 0.010s	sys 0.005s
I20260709 13:03:39.544468 17218 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:39.525877 17010 tablet_service.cc:1511] Processing CreateTablet for tablet d3c68d18d16941a9b3197081006477dc (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:39.553611 17008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdaa7a451fbb426d868eef2a6abd8849. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.562584 17219 tablet_bootstrap.cc:654] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.562491 17225 raft_consensus.cc:493] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.563184 17225 raft_consensus.cc:515] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.565488 17010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c68d18d16941a9b3197081006477dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.565727 17225 leader_election.cc:290] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:39.505267 17165 tablet_service.cc:1511] Processing CreateTablet for tablet a9bdaab91b264f3ea1e21283b3c988ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:39.558099 17220 consensus_queue.cc:260] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.571573 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:39.575075 17220 ts_tablet_manager.cc:1434] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.145s	user 0.011s	sys 0.001s
I20260709 13:03:39.576449 17220 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:39.577119 16939 tablet_service.cc:1511] Processing CreateTablet for tablet 9c455336bc164af1af57c6148b97d553 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:39.552120 17162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd447a4120b4d378b710f3f18d0541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.578629 16939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c455336bc164af1af57c6148b97d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.556896 17158 tablet_service.cc:1511] Processing CreateTablet for tablet 60c79060c3294f4fbe036c1fe48ae9bf (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 13:03:39.596197 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:03:39.578680 17066 leader_election.cc:343] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.607183 17238 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:39.607824 17238 ts_tablet_manager.cc:1403] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.142s	user 0.027s	sys 0.005s
I20260709 13:03:39.608064 17218 tablet_bootstrap.cc:654] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.608227 17089 tablet_service.cc:1511] Processing CreateTablet for tablet e14794cbc74f4fcf8d3872e3463ad58d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:39.609575 17089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14794cbc74f4fcf8d3872e3463ad58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.617225 17012 tablet_service.cc:1511] Processing CreateTablet for tablet 8f8e87277d714ded843585986eae389e (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:39.592517 17237 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:39.630352 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f8e87277d714ded843585986eae389e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.620030 17158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c79060c3294f4fbe036c1fe48ae9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.640590 17165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bdaab91b264f3ea1e21283b3c988ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.644121 17238 raft_consensus.cc:359] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:39.644959 17238 raft_consensus.cc:385] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.645557 17161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca37e8a20c2447983a81703b8706d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.645267 17238 raft_consensus.cc:740] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.669981 17156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eccff1f699a4e4387998ef0842e7d33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.683791 17238 consensus_queue.cc:260] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:39.693848 16938 tablet_service.cc:1511] Processing CreateTablet for tablet e14794cbc74f4fcf8d3872e3463ad58d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:39.708906 17219 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:39.706743 17008 tablet_service.cc:1511] Processing CreateTablet for tablet a9a2edef1b2a42bca289c34851377b55 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:39.758002 17219 ts_tablet_manager.cc:1403] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.407s	user 0.020s	sys 0.018s
I20260709 13:03:39.758322 16938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14794cbc74f4fcf8d3872e3463ad58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.717059 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:39.759163 17008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a2edef1b2a42bca289c34851377b55. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:39.759898 17067 leader_election.cc:343] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.744031 17015 tablet_service.cc:1511] Processing CreateTablet for tablet e797287dc34d42e3838f8f5a3ba68985 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:39.760442 17067 leader_election.cc:304] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:39.733155 17218 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:39.763931 17218 ts_tablet_manager.cc:1403] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.220s	user 0.020s	sys 0.008s
I20260709 13:03:39.762609 17009 tablet_service.cc:1511] Processing CreateTablet for tablet 117045952bb14176abd739933fa51259 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:39.736832 17157 tablet_service.cc:1511] Processing CreateTablet for tablet 9c455336bc164af1af57c6148b97d553 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:39.714586 17014 tablet_service.cc:1511] Processing CreateTablet for tablet afe5ed1a667143e0869e8f06a2a27686 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:39.765193 17157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c455336bc164af1af57c6148b97d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.765693 17014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe5ed1a667143e0869e8f06a2a27686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.767015 17015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e797287dc34d42e3838f8f5a3ba68985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.767642 17225 raft_consensus.cc:2749] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.694100 17160 tablet_service.cc:1511] Processing CreateTablet for tablet 391757a5f74549289d0b597a84df6740 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:39.769785 17160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391757a5f74549289d0b597a84df6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.718402 17016 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec250363e734059ae59f362df78c792 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:39.772921 17016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec250363e734059ae59f362df78c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.707185 17017 tablet_service.cc:1511] Processing CreateTablet for tablet 4633fd2f819d45d893ebcdb54511d834 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:39.775120 17009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 117045952bb14176abd739933fa51259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.781330 17164 tablet_service.cc:1511] Processing CreateTablet for tablet e14794cbc74f4fcf8d3872e3463ad58d (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:39.788650 17017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633fd2f819d45d893ebcdb54511d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.789206 17219 raft_consensus.cc:359] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.790026 17219 raft_consensus.cc:385] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.796033 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:39.799103 16918 leader_election.cc:343] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.801151 17164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14794cbc74f4fcf8d3872e3463ad58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.800292 17010 tablet_service.cc:1511] Processing CreateTablet for tablet 6a02330036da4716bc0da7d166bdea11 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:39.804536 17010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a02330036da4716bc0da7d166bdea11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.802939 17218 raft_consensus.cc:359] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:39.807523 17218 raft_consensus.cc:385] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.807861 17218 raft_consensus.cc:740] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.790326 17219 raft_consensus.cc:740] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.810230 17219 consensus_queue.cc:260] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.813089 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:39.813596 17012 tablet_service.cc:1511] Processing CreateTablet for tablet a40bdee1b5c3486b94412af27867b738 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:39.814939 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a40bdee1b5c3486b94412af27867b738. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:39.817720 16918 leader_election.cc:343] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.818219 16918 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:39.819124 17237 raft_consensus.cc:2749] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.820931 17219 ts_tablet_manager.cc:1434] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.003s
I20260709 13:03:39.818140 17218 consensus_queue.cc:260] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:39.832353 17218 ts_tablet_manager.cc:1434] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.068s	user 0.002s	sys 0.010s
I20260709 13:03:39.838128 17226 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.838732 17226 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:39.840302 17219 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:39.842897 17226 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:39.844005 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:39.844851 17220 tablet_bootstrap.cc:654] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:39.845592 17140 leader_election.cc:343] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.859943 17218 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:39.867784 17238 ts_tablet_manager.cc:1434] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.260s	user 0.024s	sys 0.015s
I20260709 13:03:39.868846 17238 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:39.865258 17219 tablet_bootstrap.cc:654] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.872265 17013 tablet_service.cc:1511] Processing CreateTablet for tablet 0395376e4b3d44a0a28a7bb199a21307 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:39.875114 17013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0395376e4b3d44a0a28a7bb199a21307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.886055 17017 tablet_service.cc:1511] Processing CreateTablet for tablet 4ed6b9a59b31479aaa96d38781de77d6 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:39.883881 17218 tablet_bootstrap.cc:654] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.894780 17220 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:39.895329 17220 ts_tablet_manager.cc:1403] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.319s	user 0.014s	sys 0.009s
I20260709 13:03:39.897748 17220 raft_consensus.cc:359] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.898479 17220 raft_consensus.cc:385] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.898782 17220 raft_consensus.cc:740] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.899526 17220 consensus_queue.cc:260] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:39.901693 17220 ts_tablet_manager.cc:1434] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:03:39.901731 17017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed6b9a59b31479aaa96d38781de77d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.903263 17220 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:39.911134 17246 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.911716 17246 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:39.915402 17246 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:39.917438 17012 tablet_service.cc:1511] Processing CreateTablet for tablet c6ce161acdcd435a8f869ae50a6ea162 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:39.938992 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ce161acdcd435a8f869ae50a6ea162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.942461 17218 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:39.943097 17218 ts_tablet_manager.cc:1403] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.006s
I20260709 13:03:39.958410 17225 raft_consensus.cc:493] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.958837 17014 tablet_service.cc:1511] Processing CreateTablet for tablet 2db96a4ed139460aa021acdd63b9b529 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:39.959147 17225 raft_consensus.cc:515] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:39.960279 17014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2db96a4ed139460aa021acdd63b9b529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:39.962761 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:39.964262 17067 leader_election.cc:343] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.967487 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:39.970055 17067 leader_election.cc:343] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.970712 17225 leader_election.cc:290] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:39.971997 17067 leader_election.cc:304] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:39.973099 17225 raft_consensus.cc:2749] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.973583 17220 tablet_bootstrap.cc:654] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.971788 17011 tablet_service.cc:1511] Processing CreateTablet for tablet edd447a4120b4d378b710f3f18d0541f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:39.973964 17218 raft_consensus.cc:359] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:39.975188 17218 raft_consensus.cc:385] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.975581 17238 tablet_bootstrap.cc:654] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.977790 17218 raft_consensus.cc:740] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.980212 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:39.974830 17011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd447a4120b4d378b710f3f18d0541f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:39.981640 17141 leader_election.cc:343] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.982112 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:39.985543 17226 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.974013 17219 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:39.986357 17219 ts_tablet_manager.cc:1403] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.146s	user 0.016s	sys 0.019s
I20260709 13:03:40.007839 17015 tablet_service.cc:1511] Processing CreateTablet for tablet a12c1a9abfa94488a9aefb37cb9ecdf0 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:40.009501 17015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12c1a9abfa94488a9aefb37cb9ecdf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.012713 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:40.014623 16993 leader_election.cc:343] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.011322 17219 raft_consensus.cc:359] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.019503 17219 raft_consensus.cc:385] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.019310 17013 tablet_service.cc:1511] Processing CreateTablet for tablet a9bdaab91b264f3ea1e21283b3c988ac (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:40.019935 17219 raft_consensus.cc:740] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.019529 17010 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17e18d23d343e6894f11dc02b2eb75 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:40.019310 17009 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2819a664db491aa04d5c6ec4c13708 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:40.007814 17016 tablet_service.cc:1511] Processing CreateTablet for tablet 100511a7ba6744e08fb2dec95a371354 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:40.020694 17013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bdaab91b264f3ea1e21283b3c988ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.021466 17218 consensus_queue.cc:260] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.026433 17225 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.027297 17225 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.032027 17225 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.033245 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.035081 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:40.036594 17066 leader_election.cc:343] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:40.037483 17067 leader_election.cc:343] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.037925 17067 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.038364 17011 tablet_service.cc:1511] Processing CreateTablet for tablet e5e1d0f0dc3240608259e975a8ea33ec (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:40.039058 17225 raft_consensus.cc:2749] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.040499 17219 consensus_queue.cc:260] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.043571 17219 ts_tablet_manager.cc:1434] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.001s
I20260709 13:03:40.045066 17219 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.044919 17015 tablet_service.cc:1511] Processing CreateTablet for tablet 391757a5f74549289d0b597a84df6740 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:40.027316 17009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2819a664db491aa04d5c6ec4c13708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.059738 17015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391757a5f74549289d0b597a84df6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.064466 17010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17e18d23d343e6894f11dc02b2eb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.066402 17220 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.066970 17220 ts_tablet_manager.cc:1403] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.164s	user 0.017s	sys 0.033s
I20260709 13:03:40.070364 17220 raft_consensus.cc:359] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.071164 17220 raft_consensus.cc:385] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.071448 17220 raft_consensus.cc:740] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.072388 17220 consensus_queue.cc:260] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.075317 17016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100511a7ba6744e08fb2dec95a371354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.075809 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.076198 17218 ts_tablet_manager.cc:1434] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.133s	user 0.006s	sys 0.006s
I20260709 13:03:40.077329 17218 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
W20260709 13:03:40.077467 16992 leader_election.cc:343] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.077986 16992 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:40.078768 17246 raft_consensus.cc:2749] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.081624 17011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e1d0f0dc3240608259e975a8ea33ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.081684 17237 raft_consensus.cc:493] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.091023 17237 raft_consensus.cc:515] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.092049 17238 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.092593 17238 ts_tablet_manager.cc:1403] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.224s	user 0.040s	sys 0.017s
I20260709 13:03:40.095676 17238 raft_consensus.cc:359] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.096426 17238 raft_consensus.cc:385] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.096725 17238 raft_consensus.cc:740] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.097555 17237 leader_election.cc:290] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.098287 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.098970 17219 tablet_bootstrap.cc:654] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.100224 17012 tablet_service.cc:1511] Processing CreateTablet for tablet 9c455336bc164af1af57c6148b97d553 (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 13:03:40.100878 16918 leader_election.cc:343] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.083794 17220 ts_tablet_manager.cc:1434] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:03:40.101781 17008 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39644 (request call id 22) took 1013 ms. Trace:
I20260709 13:03:40.102183 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.097627 17238 consensus_queue.cc:260] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:40.104054 16918 leader_election.cc:343] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.104516 16918 leader_election.cc:304] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.105353 17238 ts_tablet_manager.cc:1434] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:03:40.106431 17238 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.106559 17237 raft_consensus.cc:2749] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.107688 17017 tablet_service.cc:1511] Processing CreateTablet for tablet eb98a7c474ea41c3a428ce29909fc94f (DEFAULT_TABLE table=test-workload [id=98a69e93d6e6480e95590341f1ca437e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:40.102227 17008 rpcz_store.cc:276] 0709 13:03:39.088548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.661646 (+573098us) service_pool.cc:224] Handling call
0709 13:03:39.758560 (+ 96914us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:03:39.758713 (+   153us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:03:40.054907 (+296194us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:40.089226 (+ 34319us) tablet_replica.cc:206] Creating consensus instance
0709 13:03:40.101723 (+ 12497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.115341 17017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb98a7c474ea41c3a428ce29909fc94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.114050 17238 tablet_bootstrap.cc:654] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.102672 17220 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.122820 17219 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.123418 17219 ts_tablet_manager.cc:1403] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.000s
I20260709 13:03:40.126597 17219 raft_consensus.cc:359] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.127351 17218 tablet_bootstrap.cc:654] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.127828 17012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c455336bc164af1af57c6148b97d553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:40.137480 17219 raft_consensus.cc:385] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.138037 17219 raft_consensus.cc:740] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.140635 17226 raft_consensus.cc:493] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.141153 17226 raft_consensus.cc:515] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.145154 17220 tablet_bootstrap.cc:654] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.145435 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.144694 17219 consensus_queue.cc:260] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:40.147464 17141 leader_election.cc:343] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.149861 17219 ts_tablet_manager.cc:1434] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:40.151607 17219 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.153067 17226 leader_election.cc:290] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.154181 17246 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.154826 17246 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.158893 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:40.160847 16992 leader_election.cc:343] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.162478 17246 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.162885 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.163559 17101 raft_consensus.cc:2468] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.164328 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.164934 17141 leader_election.cc:304] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.165217 17100 raft_consensus.cc:2468] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.166214 17226 raft_consensus.cc:2804] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.166620 17226 raft_consensus.cc:493] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.166942 17226 raft_consensus.cc:3060] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.166982 16993 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:40.168692 17246 raft_consensus.cc:2804] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.169155 17246 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.169811 17246 raft_consensus.cc:3060] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.175043 17226 raft_consensus.cc:515] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.178205 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:40.179772 17141 leader_election.cc:343] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.180904 17219 tablet_bootstrap.cc:654] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.182004 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.182673 17100 raft_consensus.cc:3060] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.185489 17226 leader_election.cc:290] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.182349 17246 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.188066 17218 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.188619 17218 ts_tablet_manager.cc:1403] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.112s	user 0.023s	sys 0.018s
I20260709 13:03:40.188858 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
W20260709 13:03:40.190796 16992 leader_election.cc:343] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.192005 17238 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.193140 17238 ts_tablet_manager.cc:1403] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.087s	user 0.023s	sys 0.008s
I20260709 13:03:40.193323 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.194110 17101 raft_consensus.cc:3060] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.194509 17219 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.194998 17219 ts_tablet_manager.cc:1403] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.000s
I20260709 13:03:40.196275 17238 raft_consensus.cc:359] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.197037 17238 raft_consensus.cc:385] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.192579 17218 raft_consensus.cc:359] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.201005 17238 raft_consensus.cc:740] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.206980 17101 raft_consensus.cc:2468] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.201999 17219 raft_consensus.cc:359] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.201148 17100 raft_consensus.cc:2468] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:40.201244 17218 raft_consensus.cc:385] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.192102 17246 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.208738 17218 raft_consensus.cc:740] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.204262 17220 tablet_bootstrap.cc:492] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.208899 16993 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:40.209616 17220 ts_tablet_manager.cc:1403] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.107s	user 0.018s	sys 0.009s
I20260709 13:03:40.209775 17141 leader_election.cc:304] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.209730 17218 consensus_queue.cc:260] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.210253 17246 raft_consensus.cc:2804] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.211232 17226 raft_consensus.cc:2804] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.208252 17238 consensus_queue.cc:260] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.208261 17219 raft_consensus.cc:385] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.214346 17219 raft_consensus.cc:740] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.214788 17218 ts_tablet_manager.cc:1434] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.003s
I20260709 13:03:40.215257 17219 consensus_queue.cc:260] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.215915 17218 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.212812 17220 raft_consensus.cc:359] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.216568 17220 raft_consensus.cc:385] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.216884 17220 raft_consensus.cc:740] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.222306 17219 ts_tablet_manager.cc:1434] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.005s
I20260709 13:03:40.224071 17219 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.224596 17226 raft_consensus.cc:697] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:40.227541 17220 consensus_queue.cc:260] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.229837 17226 consensus_queue.cc:237] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.231174 17238 ts_tablet_manager.cc:1434] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.038s	user 0.012s	sys 0.000s
I20260709 13:03:40.232254 17238 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.238835 17238 tablet_bootstrap.cc:654] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.239463 17220 ts_tablet_manager.cc:1434] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:03:40.240645 17220 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.240741 17219 tablet_bootstrap.cc:654] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.227031 17246 raft_consensus.cc:697] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:40.242262 17246 consensus_queue.cc:237] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.245872 17218 tablet_bootstrap.cc:654] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.251506 17220 tablet_bootstrap.cc:654] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.253607 17219 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.254141 17219 ts_tablet_manager.cc:1403] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.012s
I20260709 13:03:40.258602 17219 raft_consensus.cc:359] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.259503 17219 raft_consensus.cc:385] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.259960 17219 raft_consensus.cc:740] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.261005 17219 consensus_queue.cc:260] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.262624 16849 catalog_manager.cc:5697] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.274363 17226 raft_consensus.cc:493] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.274915 17226 raft_consensus.cc:515] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.276852 17226 leader_election.cc:290] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.277668 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.278280 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.278463 17100 raft_consensus.cc:2468] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.279102 16952 raft_consensus.cc:2468] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.280215 17141 leader_election.cc:304] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.281965 16850 catalog_manager.cc:5697] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.283006 17218 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.283600 17218 ts_tablet_manager.cc:1403] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.068s	user 0.017s	sys 0.018s
I20260709 13:03:40.285866 17219 ts_tablet_manager.cc:1434] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:03:40.286002 17238 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.286677 17238 ts_tablet_manager.cc:1403] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.014s
I20260709 13:03:40.288014 17220 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.288615 17220 ts_tablet_manager.cc:1403] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.011s
I20260709 13:03:40.288025 17226 raft_consensus.cc:2804] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.289070 17226 raft_consensus.cc:493] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.289590 17226 raft_consensus.cc:3060] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.290185 17238 raft_consensus.cc:359] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.291041 17238 raft_consensus.cc:385] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.291416 17238 raft_consensus.cc:740] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.292222 17237 raft_consensus.cc:493] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.292009 17220 raft_consensus.cc:359] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.292928 17220 raft_consensus.cc:385] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.292790 17237 raft_consensus.cc:515] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.293279 17220 raft_consensus.cc:740] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.294116 17220 consensus_queue.cc:260] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.295213 17219 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.298359 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.295302 17237 leader_election.cc:290] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.298942 17220 ts_tablet_manager.cc:1434] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
W20260709 13:03:40.299940 16918 leader_election.cc:343] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.300436 17226 raft_consensus.cc:515] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.301188 17220 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.303973 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:40.303627 17218 raft_consensus.cc:359] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.303671 17238 consensus_queue.cc:260] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.304605 16952 raft_consensus.cc:3060] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.302645 17226 leader_election.cc:290] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.304633 17218 raft_consensus.cc:385] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.305518 17218 raft_consensus.cc:740] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.306710 17238 ts_tablet_manager.cc:1434] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:03:40.306314 17218 consensus_queue.cc:260] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.308782 17218 ts_tablet_manager.cc:1434] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:03:40.310300 17218 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.311079 17238 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.304616 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61ae0ddb123d4d8bb5b071178e546931" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.312212 17100 raft_consensus.cc:3060] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.312672 16952 raft_consensus.cc:2468] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:40.316931 17218 tablet_bootstrap.cc:654] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.318044 17140 leader_election.cc:304] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.318432 17238 tablet_bootstrap.cc:654] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.319458 17226 raft_consensus.cc:2804] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.320319 17226 raft_consensus.cc:697] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:40.321228 17226 consensus_queue.cc:237] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.323165 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:40.325549 16918 leader_election.cc:343] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.326066 16918 leader_election.cc:304] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.326877 17237 raft_consensus.cc:2749] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.327807 17220 tablet_bootstrap.cc:654] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.331771 17100 raft_consensus.cc:2468] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:40.334596 17219 tablet_bootstrap.cc:654] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.338970 17220 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.339090 17238 tablet_bootstrap.cc:492] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.339669 17238 ts_tablet_manager.cc:1403] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.005s
I20260709 13:03:40.339669 17220 ts_tablet_manager.cc:1403] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.004s
I20260709 13:03:40.343250 17220 raft_consensus.cc:359] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.344003 17220 raft_consensus.cc:385] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.344326 17220 raft_consensus.cc:740] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.345191 17220 consensus_queue.cc:260] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.347894 17220 ts_tablet_manager.cc:1434] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:40.349045 17220 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.351034 17218 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.351684 17218 ts_tablet_manager.cc:1403] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.042s	user 0.004s	sys 0.017s
I20260709 13:03:40.343101 17238 raft_consensus.cc:359] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.352186 17238 raft_consensus.cc:385] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.352545 17238 raft_consensus.cc:740] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.353483 17238 consensus_queue.cc:260] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.355027 17218 raft_consensus.cc:359] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.355839 17218 raft_consensus.cc:385] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.355942 17219 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.356043 17220 tablet_bootstrap.cc:654] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.356094 17238 ts_tablet_manager.cc:1434] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 13:03:40.356249 17218 raft_consensus.cc:740] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.356638 17219 ts_tablet_manager.cc:1403] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.001s
I20260709 13:03:40.357730 17238 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.357419 17218 consensus_queue.cc:260] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.360033 17218 ts_tablet_manager.cc:1434] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:03:40.360302 17219 raft_consensus.cc:359] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.361007 17219 raft_consensus.cc:385] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.361184 17218 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.365427 17219 raft_consensus.cc:740] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.366310 17219 consensus_queue.cc:260] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.369982 17238 tablet_bootstrap.cc:654] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.368328 17218 tablet_bootstrap.cc:654] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.385854 17219 ts_tablet_manager.cc:1434] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:03:40.387120 17238 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.387410 17219 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.387655 17238 ts_tablet_manager.cc:1403] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.004s
I20260709 13:03:40.391906 17225 raft_consensus.cc:493] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.391227 17238 raft_consensus.cc:359] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.392706 17238 raft_consensus.cc:385] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.392514 17225 raft_consensus.cc:515] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.393056 17238 raft_consensus.cc:740] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.394896 17225 leader_election.cc:290] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.394025 17238 consensus_queue.cc:260] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.395996 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e944c9ade604302b16ffd9d285523a4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.396210 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e944c9ade604302b16ffd9d285523a4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.396727 17175 raft_consensus.cc:2468] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:40.396799 17027 raft_consensus.cc:2468] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:40.398111 17067 leader_election.cc:304] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.398276 17238 ts_tablet_manager.cc:1434] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:03:40.398900 17225 raft_consensus.cc:2804] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.399256 17225 raft_consensus.cc:493] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.399538 17238 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.399636 17225 raft_consensus.cc:3060] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.395998 17219 tablet_bootstrap.cc:654] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.403602 17218 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.404167 17218 ts_tablet_manager.cc:1403] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.009s
I20260709 13:03:40.406282 16848 catalog_manager.cc:5697] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.407030 17218 raft_consensus.cc:359] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.407509 17225 raft_consensus.cc:515] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.407893 17218 raft_consensus.cc:385] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.408311 17218 raft_consensus.cc:740] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.409610 17225 leader_election.cc:290] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.411561 17226 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.409150 17218 consensus_queue.cc:260] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.414508 17226 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.416729 17218 ts_tablet_manager.cc:1434] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:03:40.417371 17246 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.417717 17218 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.418030 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.417845 17246 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.418677 17100 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.412572 17219 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.420351 17246 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.413223 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e944c9ade604302b16ffd9d285523a4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:40.417845 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.421217 17141 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.422410 17263 raft_consensus.cc:2804] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:40.422447 17140 leader_election.cc:343] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.422951 17263 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.423343 17263 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.420580 17219 ts_tablet_manager.cc:1403] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.006s
I20260709 13:03:40.424324 17218 tablet_bootstrap.cc:654] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.426460 17219 raft_consensus.cc:359] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.421471 17027 raft_consensus.cc:3060] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.427547 17219 raft_consensus.cc:385] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.427800 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.427994 17219 raft_consensus.cc:740] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.428491 16952 raft_consensus.cc:2468] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.428871 17219 consensus_queue.cc:260] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.429788 16992 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.416755 17226 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.431342 17246 raft_consensus.cc:2804] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.431733 17246 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.432063 17246 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.436698 17237 raft_consensus.cc:493] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.413688 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e944c9ade604302b16ffd9d285523a4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:40.437938 17175 raft_consensus.cc:3060] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.438084 17237 raft_consensus.cc:515] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.440563 17237 leader_election.cc:290] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.409667 17238 tablet_bootstrap.cc:654] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.443116 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.444955 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.412918 17220 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
W20260709 13:03:40.446622 16918 leader_election.cc:343] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.446813 17220 ts_tablet_manager.cc:1403] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.098s	user 0.015s	sys 0.017s
W20260709 13:03:40.447360 16918 leader_election.cc:343] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.447799 16918 leader_election.cc:304] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:40.448828 17237 raft_consensus.cc:2749] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.449662 17219 ts_tablet_manager.cc:1434] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:40.451958 17246 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.455252 17220 raft_consensus.cc:359] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.455976 17220 raft_consensus.cc:385] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.456230 17220 raft_consensus.cc:740] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.462477 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:40.463121 16952 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.463238 17219 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.466738 17246 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.456910 17220 consensus_queue.cc:260] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.475008 17219 tablet_bootstrap.cc:654] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.476030 17238 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.475801 17263 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.476613 17238 ts_tablet_manager.cc:1403] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.005s
I20260709 13:03:40.481216 17220 ts_tablet_manager.cc:1434] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:03:40.482623 17220 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.486382 16952 raft_consensus.cc:2468] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.486712 17218 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.487195 17218 ts_tablet_manager.cc:1403] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.044s
I20260709 13:03:40.487478 17175 raft_consensus.cc:2468] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:40.487872 16992 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.489344 17246 raft_consensus.cc:2804] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.489995 17218 raft_consensus.cc:359] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.490566 17263 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.490754 17218 raft_consensus.cc:385] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.491232 17218 raft_consensus.cc:740] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.490584 17246 raft_consensus.cc:697] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:40.492005 17218 consensus_queue.cc:260] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.492650 17246 consensus_queue.cc:237] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.494055 17218 ts_tablet_manager.cc:1434] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:40.495144 17218 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.495307 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:40.496111 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
W20260709 13:03:40.496559 17140 leader_election.cc:343] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.496675 17100 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.490935 17067 leader_election.cc:304] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.499258 17225 raft_consensus.cc:2804] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.498497 17027 raft_consensus.cc:2468] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:40.508481 17220 tablet_bootstrap.cc:654] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.509881 17218 tablet_bootstrap.cc:654] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.511153 17219 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.511620 17219 ts_tablet_manager.cc:1403] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.000s
I20260709 13:03:40.513803 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:40.514443 17174 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.515270 17238 raft_consensus.cc:359] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.517076 17219 raft_consensus.cc:359] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.517774 17219 raft_consensus.cc:385] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.517716 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.516005 17100 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:40.516042 17238 raft_consensus.cc:385] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.519115 17238 raft_consensus.cc:740] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.519873 17238 consensus_queue.cc:260] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.520431 17141 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:40.518210 17219 raft_consensus.cc:740] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.521592 17219 consensus_queue.cc:260] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.521981 17263 raft_consensus.cc:2804] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.522643 17263 raft_consensus.cc:697] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:40.523659 17263 consensus_queue.cc:237] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.528591 17174 raft_consensus.cc:2468] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.531731 17219 ts_tablet_manager.cc:1434] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:03:40.533036 17225 raft_consensus.cc:697] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:40.534152 17225 consensus_queue.cc:237] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.538514 17262 raft_consensus.cc:493] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.539178 17262 raft_consensus.cc:515] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.540048 17220 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.540601 17220 ts_tablet_manager.cc:1403] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.000s
I20260709 13:03:40.541754 17264 raft_consensus.cc:493] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.542124 17238 ts_tablet_manager.cc:1434] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.001s
I20260709 13:03:40.542407 17264 raft_consensus.cc:515] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.543233 17238 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.544898 17220 raft_consensus.cc:359] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.545644 17220 raft_consensus.cc:385] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.545964 17220 raft_consensus.cc:740] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.545861 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.546932 17174 raft_consensus.cc:2468] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.547101 17262 leader_election.cc:290] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.545861 17218 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.548130 17218 ts_tablet_manager.cc:1403] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.004s
I20260709 13:03:40.545889 17219 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.548517 16993 leader_election.cc:304] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.548671 17220 consensus_queue.cc:260] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.550523 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.551205 17220 ts_tablet_manager.cc:1434] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:40.552342 17220 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.553663 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.554315 16952 raft_consensus.cc:2468] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.550969 17218 raft_consensus.cc:359] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.555464 17218 raft_consensus.cc:385] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.555778 17218 raft_consensus.cc:740] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.555934 17264 leader_election.cc:290] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.556748 17264 raft_consensus.cc:2804] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.556614 17218 consensus_queue.cc:260] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.557102 17264 raft_consensus.cc:493] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.557478 17264 raft_consensus.cc:3060] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.557862 17219 tablet_bootstrap.cc:654] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.557646 16849 catalog_manager.cc:5697] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.559011 17218 ts_tablet_manager.cc:1434] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:40.559998 17218 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.564860 17238 tablet_bootstrap.cc:654] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.565662 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb0f82169da45819893f31678f63f76" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.565955 17218 tablet_bootstrap.cc:654] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.566468 17100 raft_consensus.cc:2393] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 1: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:40.567685 16993 leader_election.cc:304] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:40.568470 17262 raft_consensus.cc:3060] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.564963 17264 raft_consensus.cc:515] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.571653 17219 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.572950 17219 ts_tablet_manager.cc:1403] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.002s
I20260709 13:03:40.576683 17219 raft_consensus.cc:359] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.577584 17219 raft_consensus.cc:385] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.577901 17219 raft_consensus.cc:740] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.581023 16848 catalog_manager.cc:5697] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.578744 17219 consensus_queue.cc:260] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.582185 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:40.582785 16952 raft_consensus.cc:3060] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.586494 17219 ts_tablet_manager.cc:1434] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:40.588668 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de112b6726e84fdca382fa7b9a903c49" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:40.589497 17174 raft_consensus.cc:3060] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.581053 17264 leader_election.cc:290] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.592206 17225 raft_consensus.cc:493] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.592694 17225 raft_consensus.cc:515] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.593073 17219 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.594767 17225 leader_election.cc:290] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.596036 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.597069 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.599550 17219 tablet_bootstrap.cc:654] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:40.602067 17067 leader_election.cc:343] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:40.603013 17067 leader_election.cc:343] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.603539 17067 leader_election.cc:304] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.604489 17225 raft_consensus.cc:2749] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:40.606514 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:40.609241 16952 raft_consensus.cc:2468] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.610498 16992 leader_election.cc:304] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.611347 17264 raft_consensus.cc:2804] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.611815 17264 raft_consensus.cc:697] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:40.612669 17174 raft_consensus.cc:2468] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.612638 17264 consensus_queue.cc:237] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.620002 17220 tablet_bootstrap.cc:654] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.621492 17237 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.622118 17237 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.625661 17263 consensus_queue.cc:1048] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.627719 17218 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.628352 17218 ts_tablet_manager.cc:1403] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.069s	user 0.028s	sys 0.022s
I20260709 13:03:40.624908 17237 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.632112 17218 raft_consensus.cc:359] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.632824 17218 raft_consensus.cc:385] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.631907 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.633103 17218 raft_consensus.cc:740] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.634145 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.634421 17218 consensus_queue.cc:260] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.634822 17174 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.635520 17238 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.636175 17238 ts_tablet_manager.cc:1403] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.003s
I20260709 13:03:40.636483 17218 ts_tablet_manager.cc:1434] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:03:40.637523 17218 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.637507 17101 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.637916 17262 raft_consensus.cc:2749] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.639654 16918 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:40.641494 17237 raft_consensus.cc:2804] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.641901 17237 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.642236 17237 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.643574 17219 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.644124 17219 ts_tablet_manager.cc:1403] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.003s
I20260709 13:03:40.644766 17218 tablet_bootstrap.cc:654] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.647768 16849 catalog_manager.cc:5697] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.653672 17219 raft_consensus.cc:359] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.654640 17219 raft_consensus.cc:385] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.655000 17219 raft_consensus.cc:740] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.655892 17219 consensus_queue.cc:260] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.662819 17238 raft_consensus.cc:359] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.667778 17238 raft_consensus.cc:385] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.668150 17238 raft_consensus.cc:740] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.669028 17238 consensus_queue.cc:260] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.675139 17237 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.678016 17225 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.678607 17225 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.678912 17262 raft_consensus.cc:493] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.679414 17262 raft_consensus.cc:515] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.680658 17225 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:40.682701 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.683357 17238 ts_tablet_manager.cc:1434] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.001s
I20260709 13:03:40.684352 17238 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.685487 17262 leader_election.cc:290] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:03:40.686156 17066 leader_election.cc:343] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.685111 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.687071 17237 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:40.687821 17237 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.688252 17237 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.686030 17262 consensus_queue.cc:1048] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:40.695720 17067 leader_election.cc:343] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.708696 17219 ts_tablet_manager.cc:1434] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.064s	user 0.017s	sys 0.024s
I20260709 13:03:40.702594 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.710017 17219 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.702463 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.713403 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.715003 17218 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.715528 17218 ts_tablet_manager.cc:1403] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.004s
I20260709 13:03:40.716310 17238 tablet_bootstrap.cc:654] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.717847 17226 consensus_queue.cc:1048] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.707571 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.720027 17101 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.720095 17218 raft_consensus.cc:359] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.720849 17218 raft_consensus.cc:385] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.713430 17100 raft_consensus.cc:2468] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.721168 17218 raft_consensus.cc:740] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.722396 16993 leader_election.cc:304] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.723062 16951 raft_consensus.cc:3060] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.723453 17264 raft_consensus.cc:2804] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.723881 17264 raft_consensus.cc:493] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.724254 17264 raft_consensus.cc:3060] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.721879 17218 consensus_queue.cc:260] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.708329 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:40.725780 17174 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.710994 17067 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.727597 17303 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.727686 16952 raft_consensus.cc:2468] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:40.728200 17303 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.729439 17225 raft_consensus.cc:2749] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.734411 17218 ts_tablet_manager.cc:1434] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.006s
I20260709 13:03:40.735476 17218 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.739274 17303 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.741196 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.741422 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.742056 17218 tablet_bootstrap.cc:654] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.742182 17026 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.743873 17220 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:40.744352 17220 ts_tablet_manager.cc:1403] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.192s	user 0.027s	sys 0.017s
I20260709 13:03:40.741914 17100 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.747531 17220 raft_consensus.cc:359] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.745237 17101 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:40.758500 17220 raft_consensus.cc:385] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.751942 16847 catalog_manager.cc:5697] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.758867 17220 raft_consensus.cc:740] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.759593 17220 consensus_queue.cc:260] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.746045 16918 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.761623 17220 ts_tablet_manager.cc:1434] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 13:03:40.762537 17220 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:40.762573 16918 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:40.763484 17303 raft_consensus.cc:2804] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.763831 17303 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.764151 17303 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.757946 17174 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:40.777945 17264 raft_consensus.cc:515] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.783015 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:40.783105 17237 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.783651 16952 raft_consensus.cc:3060] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.785648 17219 tablet_bootstrap.cc:654] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.787446 17262 consensus_queue.cc:1048] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.794878 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.798702 17218 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.799464 17218 ts_tablet_manager.cc:1403] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.011s
I20260709 13:03:40.800348 17238 tablet_bootstrap.cc:492] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.801028 17238 ts_tablet_manager.cc:1403] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.117s	user 0.008s	sys 0.004s
I20260709 13:03:40.804795 17219 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:40.805590 17219 ts_tablet_manager.cc:1403] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.000s
I20260709 13:03:40.806991 17237 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.807576 17237 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.809975 17237 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:40.810645 17219 raft_consensus.cc:359] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.811435 17219 raft_consensus.cc:385] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.811426 16918 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.811975 17219 raft_consensus.cc:740] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.811901 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.812446 17303 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.813601 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.814391 17174 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.812716 17026 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:40.815879 16918 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.816665 17219 consensus_queue.cc:260] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.820348 17237 raft_consensus.cc:2749] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.817057 16952 raft_consensus.cc:2468] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.831720 17238 raft_consensus.cc:359] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.832571 17238 raft_consensus.cc:385] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.833411 17238 raft_consensus.cc:740] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.832971 17302 raft_consensus.cc:2804] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.834766 17264 leader_election.cc:290] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.828634 17218 raft_consensus.cc:359] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.836259 17218 raft_consensus.cc:385] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.837143 17218 raft_consensus.cc:740] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.829962 17220 tablet_bootstrap.cc:654] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.842712 17218 consensus_queue.cc:260] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.831274 16992 leader_election.cc:304] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:40.850737 17218 ts_tablet_manager.cc:1434] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.002s
I20260709 13:03:40.851552 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c2977ecbd943b3bbffc28c746df99f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.851773 17218 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:40.852133 17101 raft_consensus.cc:3060] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.855108 17238 consensus_queue.cc:260] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.870604 17219 ts_tablet_manager.cc:1434] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.004s
I20260709 13:03:40.872540 17101 raft_consensus.cc:2468] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:40.873466 17218 tablet_bootstrap.cc:654] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.859362 17226 raft_consensus.cc:493] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.863883 17264 raft_consensus.cc:2804] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.874058 17238 ts_tablet_manager.cc:1434] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 13:03:40.874874 17219 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:40.880877 17303 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.881225 17225 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.881508 17264 raft_consensus.cc:697] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:40.881767 17225 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.882421 17264 consensus_queue.cc:237] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.883846 17225 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:40.879832 17226 raft_consensus.cc:515] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.888407 17226 leader_election.cc:290] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.875283 17302 raft_consensus.cc:697] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:40.890879 17237 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.891741 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:40.869984 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 53639.1us
I20260709 13:03:40.897497 17237 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:40.907054 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.907392 17238 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.907804 17027 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.909904 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:40.910604 17101 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:40.911234 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:40.913018 17067 leader_election.cc:343] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.914583 17141 leader_election.cc:304] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:40.914826 17237 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:40.915306 17263 raft_consensus.cc:2804] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.915684 17263 raft_consensus.cc:493] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.916051 17263 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.915884 17226 consensus_queue.cc:1048] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.916262 17238 tablet_bootstrap.cc:654] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.920464 17226 consensus_queue.cc:1048] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:40.922451 17066 leader_election.cc:343] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.922892 17066 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:40.923956 17264 consensus_queue.cc:1048] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.930431 17225 raft_consensus.cc:2749] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:40.935947 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:40.936775 17027 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.942184 17218 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:40.943066 17218 ts_tablet_manager.cc:1403] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.002s
I20260709 13:03:40.943740 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.944388 17101 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.948870 17218 raft_consensus.cc:359] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.949606 17218 raft_consensus.cc:385] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.949872 17218 raft_consensus.cc:740] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.951372 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:40.951277 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:40.951946 17263 raft_consensus.cc:515] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:40.952497 17175 raft_consensus.cc:2468] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
W20260709 13:03:40.953030 16918 leader_election.cc:343] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:40.953923 16918 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:40.940798 17302 consensus_queue.cc:237] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.955112 16850 catalog_manager.cc:5697] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.957583 17027 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:40.954090 17263 leader_election.cc:290] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:40.956964 17218 consensus_queue.cc:260] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:40.958530 17226 consensus_queue.cc:1048] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.960541 17219 tablet_bootstrap.cc:654] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.960390 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:40.961107 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:40.968286 17026 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.968998 17238 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:40.963526 17320 raft_consensus.cc:2804] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.955148 17314 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.964125 17264 consensus_queue.cc:1048] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.967754 17098 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.969583 17238 ts_tablet_manager.cc:1403] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.003s
I20260709 13:03:40.970468 17314 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.973032 17314 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.977025 17238 raft_consensus.cc:359] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.977972 17238 raft_consensus.cc:385] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:40.978256 17238 raft_consensus.cc:740] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:40.979782 17238 consensus_queue.cc:260] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.984935 17218 ts_tablet_manager.cc:1434] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:03:40.986629 17225 raft_consensus.cc:493] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.987121 17225 raft_consensus.cc:515] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:40.988973 17238 ts_tablet_manager.cc:1434] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:03:40.990058 17238 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:40.993470 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:40.994168 16950 raft_consensus.cc:2468] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:40.996559 17238 tablet_bootstrap.cc:654] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:40.997313 17066 leader_election.cc:304] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:40.998399 17289 raft_consensus.cc:2804] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.998788 17289 raft_consensus.cc:493] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.999109 17289 raft_consensus.cc:3060] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.989171 17225 leader_election.cc:290] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:40.971326 17320 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.007697 17320 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.006753 16918 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:41.020154 17319 raft_consensus.cc:2804] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.020789 17319 raft_consensus.cc:697] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:41.021915 17319 consensus_queue.cc:237] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.016624 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:41.024926 17027 raft_consensus.cc:2468] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:41.027158 17218 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.031946 17225 consensus_queue.cc:1048] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.039204 17220 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.039656 17220 ts_tablet_manager.cc:1403] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.277s	user 0.012s	sys 0.004s
I20260709 13:03:41.042873 17220 raft_consensus.cc:359] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.043542 17220 raft_consensus.cc:385] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.043658 17218 tablet_bootstrap.cc:654] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.043843 17220 raft_consensus.cc:740] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.044720 17220 consensus_queue.cc:260] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.036439 17026 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.037159 17101 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:41.050719 17220 ts_tablet_manager.cc:1434] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:03:41.052850 17220 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.059710 17141 leader_election.cc:304] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:41.060833 17263 raft_consensus.cc:2804] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.061339 17263 raft_consensus.cc:697] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:41.057694 17289 raft_consensus.cc:515] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.067260 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:41.067816 16950 raft_consensus.cc:3060] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.068813 17238 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.069468 17238 ts_tablet_manager.cc:1403] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.022s
I20260709 13:03:41.062144 17263 consensus_queue.cc:237] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.086066 17098 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.067806 17314 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.088857 17314 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.089496 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:41.090117 17174 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.090665 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:41.091467 17026 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.097899 17220 tablet_bootstrap.cc:654] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.113138 16950 raft_consensus.cc:2468] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:41.114786 17066 leader_election.cc:304] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:41.122570 17219 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:41.123304 17219 ts_tablet_manager.cc:1403] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.249s	user 0.010s	sys 0.008s
I20260709 13:03:41.101452 17289 leader_election.cc:290] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.125698 16849 catalog_manager.cc:5697] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.153851 17238 raft_consensus.cc:359] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.154613 17238 raft_consensus.cc:385] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.154865 17238 raft_consensus.cc:740] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.152781 17174 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:41.161167 17264 consensus_queue.cc:1048] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.162410 16918 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:41.163255 17314 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.163726 17314 raft_consensus.cc:697] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:41.164489 17314 consensus_queue.cc:237] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.168052 17220 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.168538 17220 ts_tablet_manager.cc:1403] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.117s	user 0.010s	sys 0.001s
I20260709 13:03:41.170456 16950 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.168979 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ed0bc1914444d179b36b1253f15c65c" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:41.171442 17220 raft_consensus.cc:359] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.171957 17025 raft_consensus.cc:3060] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.172224 17220 raft_consensus.cc:385] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.172610 17220 raft_consensus.cc:740] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.173462 17238 consensus_queue.cc:260] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.176546 17238 ts_tablet_manager.cc:1434] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.107s	user 0.006s	sys 0.002s
I20260709 13:03:41.176376 17220 consensus_queue.cc:260] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.177587 17238 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.168968 17219 raft_consensus.cc:359] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.178678 17220 ts_tablet_manager.cc:1434] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:03:41.179837 17220 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.180753 16847 catalog_manager.cc:5697] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.178737 17219 raft_consensus.cc:385] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.184990 17219 raft_consensus.cc:740] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.189924 17219 consensus_queue.cc:260] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.198627 17218 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.199254 17218 ts_tablet_manager.cc:1403] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.172s	user 0.028s	sys 0.014s
I20260709 13:03:41.200424 17341 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.192138 17289 consensus_queue.cc:1048] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:03:41.208312 17219 ts_tablet_manager.cc:1434] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.001s
I20260709 13:03:41.209308 17219 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:41.201087 17341 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.214769 17220 tablet_bootstrap.cc:654] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:41.191047 17066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 76349.3us
I20260709 13:03:41.212631 17246 consensus_queue.cc:1048] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:03:41.226811 17238 tablet_bootstrap.cc:654] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:41.197697 17067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 89323us
I20260709 13:03:41.229916 17319 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.235072 17319 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.238489 17341 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:41.238602 17026 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:41.239449 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:41.241279 17320 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.241828 17173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:41.243186 17219 tablet_bootstrap.cc:654] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.237128 17319 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:41.248189 17263 consensus_queue.cc:1048] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:41.253816 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:41.255005 17338 raft_consensus.cc:2804] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.255846 16918 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:41.255741 17338 raft_consensus.cc:697] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:41.257179 17302 raft_consensus.cc:2749] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:41.258114 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:41.258224 17338 consensus_queue.cc:237] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:41.258330 17067 leader_election.cc:343] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:41.265594 17066 leader_election.cc:343] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.268293 17066 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:41.271730 17320 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.282675 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:41.286849 16850 catalog_manager.cc:5697] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.282985 17025 raft_consensus.cc:2468] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:41.295575 16850 catalog_manager.cc:5697] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.297266 17218 raft_consensus.cc:359] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.297983 17218 raft_consensus.cc:385] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.298242 17218 raft_consensus.cc:740] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.298897 17218 consensus_queue.cc:260] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.300969 17218 ts_tablet_manager.cc:1434] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.101s	user 0.005s	sys 0.000s
I20260709 13:03:41.301865 17218 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.293948 17172 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.291802 17341 raft_consensus.cc:2749] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.307011 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:41.318277 17342 raft_consensus.cc:493] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.318708 16918 leader_election.cc:343] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.318946 17342 raft_consensus.cc:515] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.321168 17246 consensus_queue.cc:1048] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.333992 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:41.337379 17141 leader_election.cc:343] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.343899 17342 leader_election.cc:290] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:41.345366 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:41.346091 16950 raft_consensus.cc:2468] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:41.347735 17264 consensus_queue.cc:1048] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.347955 17172 raft_consensus.cc:2468] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:41.349104 17219 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:41.347748 17140 leader_election.cc:304] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:03:41.352422 17238 tablet_bootstrap.cc:492] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.353681 16918 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.354909 17320 raft_consensus.cc:2804] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.355370 17320 raft_consensus.cc:697] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:41.355463 16850 catalog_manager.cc:5697] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.356217 17320 consensus_queue.cc:237] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.360203 17342 raft_consensus.cc:2804] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.360745 17342 raft_consensus.cc:493] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.353050 17238 ts_tablet_manager.cc:1403] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.013s
I20260709 13:03:41.361126 17342 raft_consensus.cc:3060] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.364714 17218 tablet_bootstrap.cc:654] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.362484 17219 ts_tablet_manager.cc:1403] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.153s	user 0.017s	sys 0.006s
I20260709 13:03:41.369678 17343 raft_consensus.cc:493] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.370293 17343 raft_consensus.cc:515] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.373533 17343 leader_election.cc:290] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.375375 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:41.379792 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:41.380834 17100 raft_consensus.cc:2468] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:41.382771 17141 leader_election.cc:304] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:41.384078 17343 raft_consensus.cc:2804] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.384786 17343 raft_consensus.cc:493] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.385403 17343 raft_consensus.cc:3060] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.390054 17320 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.390684 17320 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.391124 17238 raft_consensus.cc:359] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.391850 17238 raft_consensus.cc:385] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.392109 17238 raft_consensus.cc:740] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.392791 17220 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.393460 17220 ts_tablet_manager.cc:1403] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.214s	user 0.022s	sys 0.006s
I20260709 13:03:41.395429 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:41.397012 16918 leader_election.cc:343] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.378862 17219 raft_consensus.cc:359] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.398697 17219 raft_consensus.cc:385] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.398944 17219 raft_consensus.cc:740] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.399581 17219 consensus_queue.cc:260] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.392763 17238 consensus_queue.cc:260] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.403664 17026 raft_consensus.cc:2468] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:41.405706 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:41.410610 17219 ts_tablet_manager.cc:1434] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:03:41.411587 17219 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:41.397702 17320 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.403177 17220 raft_consensus.cc:359] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.416524 17220 raft_consensus.cc:385] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.416852 17220 raft_consensus.cc:740] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.417644 17220 consensus_queue.cc:260] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.418548 17174 raft_consensus.cc:2468] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:41.420128 17343 raft_consensus.cc:515] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.420393 16918 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.426492 17238 ts_tablet_manager.cc:1434] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 13:03:41.422751 17320 raft_consensus.cc:2804] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.427870 17220 ts_tablet_manager.cc:1434] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:03:41.427953 17238 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.429538 17220 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.433399 17219 tablet_bootstrap.cc:654] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.428771 17320 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.439073 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:41.423470 17342 raft_consensus.cc:515] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.440202 17220 tablet_bootstrap.cc:654] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.442194 17026 raft_consensus.cc:3060] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.441869 17218 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.444918 17218 ts_tablet_manager.cc:1403] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.143s	user 0.038s	sys 0.020s
I20260709 13:03:41.447506 16850 catalog_manager.cc:5697] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.447944 17218 raft_consensus.cc:359] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.448653 17218 raft_consensus.cc:385] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.448925 17218 raft_consensus.cc:740] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.449339 17320 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.449602 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:41.451694 17342 leader_election.cc:290] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:41.453538 17343 leader_election.cc:290] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.453732 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1ffde0abe4f029d91b01b8d08d2b7" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:41.463201 16950 raft_consensus.cc:3060] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.454856 17319 consensus_queue.cc:1048] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.456844 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8c7883a65740a5842d54e35490a6e1" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:41.458577 17026 raft_consensus.cc:2468] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.467945 17098 raft_consensus.cc:3060] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.449625 17218 consensus_queue.cc:260] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.469249 17100 raft_consensus.cc:3060] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.477346 17100 raft_consensus.cc:2468] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.477378 17098 raft_consensus.cc:2468] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.485670 17141 leader_election.cc:304] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:41.486580 17263 raft_consensus.cc:2804] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.487112 17263 raft_consensus.cc:697] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:41.489121 17263 consensus_queue.cc:237] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.490149 16950 raft_consensus.cc:2468] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:41.491534 17140 leader_election.cc:304] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:41.495465 17320 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.492702 17343 raft_consensus.cc:2804] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.496706 17343 raft_consensus.cc:697] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:41.497761 17320 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.497620 17343 consensus_queue.cc:237] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.499318 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:41.499934 17172 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.501479 17218 ts_tablet_manager.cc:1434] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.004s
I20260709 13:03:41.502468 17218 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.509688 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:41.514011 16918 leader_election.cc:343] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.514000 17218 tablet_bootstrap.cc:654] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.518962 17172 raft_consensus.cc:2468] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:41.521706 16918 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.522614 17238 tablet_bootstrap.cc:654] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.523048 17314 raft_consensus.cc:2804] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.523598 17314 raft_consensus.cc:697] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:41.524467 17314 consensus_queue.cc:237] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.532388 17219 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:41.533004 17219 ts_tablet_manager.cc:1403] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.011s
I20260709 13:03:41.533625 17237 consensus_queue.cc:1048] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.537315 17220 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.537809 17220 ts_tablet_manager.cc:1403] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.108s	user 0.024s	sys 0.023s
I20260709 13:03:41.548951 17341 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.549541 17341 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.551656 17341 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.553256 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:41.554831 17067 leader_election.cc:343] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.556281 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:41.557734 17066 leader_election.cc:343] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.558192 17066 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.559998 17220 raft_consensus.cc:359] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.560695 17220 raft_consensus.cc:385] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.560796 17341 raft_consensus.cc:2749] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.565228 17219 raft_consensus.cc:359] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.565907 17219 raft_consensus.cc:385] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.566149 17219 raft_consensus.cc:740] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.566751 17219 consensus_queue.cc:260] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.561000 17220 raft_consensus.cc:740] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.568048 17220 consensus_queue.cc:260] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.570442 17220 ts_tablet_manager.cc:1434] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:03:41.571725 17219 ts_tablet_manager.cc:1434] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.003s
I20260709 13:03:41.572947 17219 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:41.573819 16850 catalog_manager.cc:5697] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.575012 17320 consensus_queue.cc:1048] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.577692 17220 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.578267 16850 catalog_manager.cc:5697] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.587569 17219 tablet_bootstrap.cc:654] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.588913 17218 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.589419 17218 ts_tablet_manager.cc:1403] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.034s
I20260709 13:03:41.593637 17320 consensus_queue.cc:1048] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.597759 17238 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.597043 16848 catalog_manager.cc:5697] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.598266 17238 ts_tablet_manager.cc:1403] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.171s	user 0.025s	sys 0.024s
I20260709 13:03:41.601258 17238 raft_consensus.cc:359] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.605057 17238 raft_consensus.cc:385] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.605437 17238 raft_consensus.cc:740] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.606290 17238 consensus_queue.cc:260] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.608361 17238 ts_tablet_manager.cc:1434] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 13:03:41.609256 17238 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.610641 17302 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.603643 17218 raft_consensus.cc:359] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.614104 17218 raft_consensus.cc:385] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.614378 17218 raft_consensus.cc:740] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.615348 17218 consensus_queue.cc:260] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.616765 17264 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.617333 17264 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.617990 17218 ts_tablet_manager.cc:1434] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:03:41.620189 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:41.620548 17238 tablet_bootstrap.cc:654] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.620903 16950 raft_consensus.cc:2468] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:41.622349 16992 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:41.623230 17246 raft_consensus.cc:2804] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.623739 17246 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.624148 17246 raft_consensus.cc:3060] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.625049 17314 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.625751 17314 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.628264 17314 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.629395 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:41.629977 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:41.631407 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:41.630172 17098 raft_consensus.cc:2468] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:41.620316 17264 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.633111 16918 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:41.634742 17218 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.635277 17314 raft_consensus.cc:2804] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.635805 17314 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.636256 17314 raft_consensus.cc:3060] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.637616 17246 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.638912 17027 raft_consensus.cc:2393] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:41.640093 17343 consensus_queue.cc:1048] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.641420 17218 tablet_bootstrap.cc:654] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.643658 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:41.645612 17246 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.647606 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:41.648236 17100 raft_consensus.cc:3060] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.648912 17220 tablet_bootstrap.cc:654] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.655373 17238 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.656046 17238 ts_tablet_manager.cc:1403] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.018s
I20260709 13:03:41.666493 17238 raft_consensus.cc:359] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.669494 17218 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.670524 17218 ts_tablet_manager.cc:1403] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.002s
I20260709 13:03:41.669451 17343 consensus_queue.cc:1048] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.679854 17218 raft_consensus.cc:359] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.680581 17218 raft_consensus.cc:385] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.680850 17218 raft_consensus.cc:740] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.681504 17218 consensus_queue.cc:260] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.684823 17238 raft_consensus.cc:385] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.685159 17237 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.685858 17218 ts_tablet_manager.cc:1434] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:41.686846 17218 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.685170 17238 raft_consensus.cc:740] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.689512 17238 consensus_queue.cc:260] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:41.694164 17238 ts_tablet_manager.cc:1434] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:03:41.686208 17314 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.695250 16950 raft_consensus.cc:2393] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 1: Already voted for candidate 64734f14219348f59d1cf8700416565c in this term.
I20260709 13:03:41.695333 17218 tablet_bootstrap.cc:654] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.700618 17219 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:41.701926 17219 ts_tablet_manager.cc:1403] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.129s	user 0.022s	sys 0.011s
I20260709 13:03:41.695250 17238 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.710004 17219 raft_consensus.cc:359] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.710692 17219 raft_consensus.cc:385] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.710932 17219 raft_consensus.cc:740] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.711567 17219 consensus_queue.cc:260] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.703737 17314 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.701469 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:41.714716 17219 ts_tablet_manager.cc:1434] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:41.715057 17027 raft_consensus.cc:2393] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:41.703109 17220 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.715968 17220 ts_tablet_manager.cc:1403] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.002s
I20260709 13:03:41.704640 17218 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.716243 17219 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:41.716641 17218 ts_tablet_manager.cc:1403] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.005s
I20260709 13:03:41.718988 17220 raft_consensus.cc:359] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.704962 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "006cb6681bb949508a3ac253c5a64661" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
W20260709 13:03:41.704221 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:41.719659 17218 raft_consensus.cc:359] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.701231 17100 raft_consensus.cc:2468] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:41.720762 17218 raft_consensus.cc:385] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.721086 17218 raft_consensus.cc:740] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.719815 17220 raft_consensus.cc:385] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.721853 17218 consensus_queue.cc:260] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.722091 16918 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.722404 17220 raft_consensus.cc:740] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.723922 17238 tablet_bootstrap.cc:654] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.725699 17218 ts_tablet_manager.cc:1434] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:03:41.726027 17339 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.725472 17314 raft_consensus.cc:2749] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.726082 16993 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:41.726701 17339 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.727638 17246 raft_consensus.cc:2804] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:41.728145 17246 raft_consensus.cc:697] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:41.728847 17339 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:41.728983 17246 consensus_queue.cc:237] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.729988 17218 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.730997 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:41.731004 17220 consensus_queue.cc:260] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.732401 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:41.732758 17067 leader_election.cc:343] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.734345 17220 ts_tablet_manager.cc:1434] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
W20260709 13:03:41.734555 17067 leader_election.cc:343] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.735023 17067 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.735381 17220 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.736596 17339 raft_consensus.cc:2749] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.744035 17238 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.744611 17238 ts_tablet_manager.cc:1403] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.000s
I20260709 13:03:41.744632 17218 tablet_bootstrap.cc:654] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.747521 17238 raft_consensus.cc:359] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.748234 17238 raft_consensus.cc:385] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.748534 17238 raft_consensus.cc:740] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.749208 17238 consensus_queue.cc:260] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.752084 17238 ts_tablet_manager.cc:1434] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:41.753332 17238 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.753634 17339 consensus_queue.cc:1048] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.758177 17220 tablet_bootstrap.cc:654] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.762380 17218 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.762849 17218 ts_tablet_manager.cc:1403] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.000s
I20260709 13:03:41.763762 17238 tablet_bootstrap.cc:654] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.765905 17218 raft_consensus.cc:359] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.766876 17218 raft_consensus.cc:385] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.767232 17218 raft_consensus.cc:740] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.768272 17218 consensus_queue.cc:260] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.770893 17218 ts_tablet_manager.cc:1434] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:41.772127 17220 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:41.772666 17220 ts_tablet_manager.cc:1403] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.001s
I20260709 13:03:41.774206 17218 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.775338 17220 raft_consensus.cc:359] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.781998 17220 raft_consensus.cc:385] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.782378 17220 raft_consensus.cc:740] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.786019 17238 tablet_bootstrap.cc:492] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:41.786671 17238 ts_tablet_manager.cc:1403] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.005s
I20260709 13:03:41.788437 17219 tablet_bootstrap.cc:654] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.789582 17218 tablet_bootstrap.cc:654] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.792109 17238 raft_consensus.cc:359] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.792904 17238 raft_consensus.cc:385] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.794610 17238 raft_consensus.cc:740] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.795408 17238 consensus_queue.cc:260] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.796108 17220 consensus_queue.cc:260] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.797863 17238 ts_tablet_manager.cc:1434] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:41.798934 17238 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:41.799966 17218 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.800647 17218 ts_tablet_manager.cc:1403] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:03:41.805622 17238 tablet_bootstrap.cc:654] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.807080 17027 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.816751 17220 ts_tablet_manager.cc:1434] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 13:03:41.818274 17220 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:41.819883 17314 consensus_queue.cc:1048] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.855997 17338 raft_consensus.cc:493] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.856711 17338 raft_consensus.cc:515] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.859663 17338 leader_election.cc:290] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:41.860306 17338 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.864436 17341 consensus_queue.cc:1048] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.867585 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:41.873972 17067 leader_election.cc:343] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.879015 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:41.880620 17066 leader_election.cc:343] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.881251 17066 leader_election.cc:304] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:41.882753 17341 raft_consensus.cc:2749] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.892666 17314 consensus_queue.cc:1048] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.860893 17338 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.903285 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:41.905838 17067 leader_election.cc:343] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.908456 17314 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.909194 17314 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:41.909574 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:41.911720 17066 leader_election.cc:343] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:41.911952 17314 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:41.912379 17066 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:41.917660 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:41.918749 17024 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:41.920686 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:41.921744 17098 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:41.931373 17219 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:41.932119 17219 ts_tablet_manager.cc:1403] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.216s	user 0.006s	sys 0.006s
I20260709 13:03:41.932626 17343 consensus_queue.cc:1048] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:41.933718 16918 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:41.937147 17314 raft_consensus.cc:2804] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:41.937677 17314 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.938539 17314 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:41.947788 17341 raft_consensus.cc:2749] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:41.948637 17338 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:41.953572 17218 raft_consensus.cc:359] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.962544 17218 raft_consensus.cc:385] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.965993 17218 raft_consensus.cc:740] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.967031 17218 consensus_queue.cc:260] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:41.954562 17237 consensus_queue.cc:1048] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:41.971670 17218 ts_tablet_manager.cc:1434] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.171s	user 0.007s	sys 0.002s
I20260709 13:03:41.973042 17218 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:41.982052 17218 tablet_bootstrap.cc:654] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:41.993835 17219 raft_consensus.cc:359] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:41.994731 17219 raft_consensus.cc:385] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:41.995142 17219 raft_consensus.cc:740] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:41.995602 17218 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:41.996415 17218 ts_tablet_manager.cc:1403] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 13:03:42.000213 17218 raft_consensus.cc:359] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.001402 17218 raft_consensus.cc:385] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.001832 17218 raft_consensus.cc:740] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.002766 17218 consensus_queue.cc:260] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.005728 17218 ts_tablet_manager.cc:1434] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:03:42.007077 17218 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:42.009191 17343 raft_consensus.cc:493] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.009861 17343 raft_consensus.cc:515] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.011597 17220 tablet_bootstrap.cc:654] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.012261 17343 leader_election.cc:290] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:42.014277 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.014430 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.015015 17098 raft_consensus.cc:2468] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
W20260709 13:03:42.015748 17141 leader_election.cc:343] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.016759 17141 leader_election.cc:304] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.018445 17343 raft_consensus.cc:2804] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.018971 17343 raft_consensus.cc:493] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.019400 17343 raft_consensus.cc:3060] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.020627 17237 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.026154 17237 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.015201 17218 tablet_bootstrap.cc:654] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.028324 17237 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.028859 17343 raft_consensus.cc:515] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.029764 17238 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:42.031044 17238 ts_tablet_manager.cc:1403] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.232s	user 0.094s	sys 0.105s
I20260709 13:03:42.031266 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.031901 17263 consensus_queue.cc:1048] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.033119 17343 leader_election.cc:290] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:42.034588 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.034744 17238 raft_consensus.cc:359] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.035434 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:42.035784 17238 raft_consensus.cc:385] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.036091 17098 raft_consensus.cc:3060] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.036301 17238 raft_consensus.cc:740] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.037369 17238 consensus_queue.cc:260] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.041656 17218 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:42.042251 17218 ts_tablet_manager.cc:1403] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.004s
I20260709 13:03:42.042423 17238 ts_tablet_manager.cc:1434] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:03:42.042878 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.043547 17238 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:42.044708 17098 raft_consensus.cc:2468] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:42.045526 17218 raft_consensus.cc:359] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:42.046591 16918 consensus_peers.cc:597] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:42.047370 16918 leader_election.cc:343] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.047587 17141 leader_election.cc:304] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:42.048689 17343 raft_consensus.cc:2804] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.049368 17343 raft_consensus.cc:697] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:42.050612 17219 consensus_queue.cc:260] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.050359 17343 consensus_queue.cc:237] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.051528 17238 tablet_bootstrap.cc:654] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:42.051679 17141 leader_election.cc:343] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.046288 17218 raft_consensus.cc:385] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.054868 17218 raft_consensus.cc:740] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.056146 17218 consensus_queue.cc:260] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.060783 17319 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.061424 17319 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.062820 17343 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.063606 17343 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.064229 17264 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.064736 17264 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.066799 17343 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:42.068729 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.066816 17264 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:42.070026 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.070562 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:42.071827 16993 leader_election.cc:343] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:42.072606 16992 leader_election.cc:343] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.073022 16992 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
W20260709 13:03:42.071827 17140 leader_election.cc:343] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.074828 17264 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.074905 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.075783 17098 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:42.060792 17218 ts_tablet_manager.cc:1434] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:03:42.076963 17141 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:42.081175 17263 consensus_queue.cc:1048] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.082561 17263 raft_consensus.cc:2804] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.082908 17263 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.083245 17263 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.093434 17319 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.093587 17343 consensus_queue.cc:1048] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.094115 17218 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:42.115726 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.116052 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.109704 17264 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.111249 17219 ts_tablet_manager.cc:1434] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.179s	user 0.010s	sys 0.001s
I20260709 13:03:42.118232 17264 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.120608 17218 tablet_bootstrap.cc:654] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.120662 17219 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:42.121783 17264 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.124437 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.125197 17098 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:42.126655 16993 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:42.127626 17264 raft_consensus.cc:2804] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.118177 17100 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:42.108403 17238 tablet_bootstrap.cc:492] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:42.112365 17025 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:42.129576 17238 ts_tablet_manager.cc:1403] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.086s	user 0.027s	sys 0.032s
I20260709 13:03:42.103231 17263 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.131776 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.132970 17174 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:42.133893 17263 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:42.134909 17238 raft_consensus.cc:359] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.134891 16918 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:42.131580 17219 tablet_bootstrap.cc:654] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.137349 17238 raft_consensus.cc:385] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.137746 17238 raft_consensus.cc:740] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.138126 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:42.138468 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:42.130584 17264 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.139497 17264 raft_consensus.cc:3060] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:42.140393 17140 leader_election.cc:343] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.138475 17238 consensus_queue.cc:260] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.138792 17098 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.145612 17238 ts_tablet_manager.cc:1434] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:03:42.146365 17302 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.147624 17302 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.148020 17302 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.148540 17238 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
W20260709 13:03:42.146890 16918 leader_election.cc:343] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.154132 17314 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.161592 17220 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:42.162205 17220 ts_tablet_manager.cc:1403] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.344s	user 0.010s	sys 0.005s
I20260709 13:03:42.164306 16918 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.165741 17237 raft_consensus.cc:2804] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.166158 17237 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.165516 16850 catalog_manager.cc:5697] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.166510 17237 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.172446 17098 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:42.172331 17302 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.171767 16848 catalog_manager.cc:5697] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:42.175580 17218 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:42.176352 17218 ts_tablet_manager.cc:1403] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.010s
I20260709 13:03:42.177647 17141 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:42.179322 17263 raft_consensus.cc:2804] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.180070 17263 raft_consensus.cc:697] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:42.194255 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:42.193532 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:42.212812 17101 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.190373 17264 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.191589 17246 consensus_queue.cc:1048] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:03:42.187187 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.214717 17025 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.216889 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
W20260709 13:03:42.217121 16918 leader_election.cc:343] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.217559 17100 raft_consensus.cc:3060] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.180940 17263 consensus_queue.cc:237] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.190120 17220 raft_consensus.cc:359] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.222172 17220 raft_consensus.cc:385] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.193856 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:42.222584 17220 raft_consensus.cc:740] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.222996 17098 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.223345 17220 consensus_queue.cc:260] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.180510 17238 tablet_bootstrap.cc:654] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.225114 17100 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:42.191847 17302 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.209077 17218 raft_consensus.cc:359] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.226514 16993 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:42.226817 17218 raft_consensus.cc:385] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.227685 17218 raft_consensus.cc:740] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.228865 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:42.228070 17246 raft_consensus.cc:2804] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.229513 17174 raft_consensus.cc:3060] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.226480 17025 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.196048 17237 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.230091 17246 raft_consensus.cc:697] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:42.231262 17246 consensus_queue.cc:237] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.185074 17314 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:42.233088 17220 ts_tablet_manager.cc:1434] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.003s
I20260709 13:03:42.246712 17220 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:42.251228 17262 consensus_queue.cc:1048] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.232890 17218 consensus_queue.cc:260] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.234457 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:42.237906 16918 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:42.253443 17314 raft_consensus.cc:2804] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.253931 17314 raft_consensus.cc:697] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
W20260709 13:03:42.253973 16918 leader_election.cc:343] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.254927 17218 ts_tablet_manager.cc:1434] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 13:03:42.254825 17314 consensus_queue.cc:237] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.255426 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.256155 17025 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.257956 17218 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:42.239233 17237 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.241014 17238 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:42.268095 17238 ts_tablet_manager.cc:1403] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.000s
I20260709 13:03:42.241820 17219 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:42.269140 17219 ts_tablet_manager.cc:1403] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.149s	user 0.007s	sys 0.010s
I20260709 13:03:42.244622 17264 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.274214 17218 tablet_bootstrap.cc:654] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.282799 17219 raft_consensus.cc:359] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.283370 17219 raft_consensus.cc:385] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.283602 17219 raft_consensus.cc:740] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.282500 17101 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.285542 17025 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.286779 17238 raft_consensus.cc:359] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.287482 17238 raft_consensus.cc:385] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.287755 17238 raft_consensus.cc:740] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.288416 16918 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.289278 17238 consensus_queue.cc:260] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.291112 17098 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.292834 17238 ts_tablet_manager.cc:1434] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:03:42.293757 17238 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:42.293892 17219 consensus_queue.cc:260] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.298262 17220 tablet_bootstrap.cc:654] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.293485 17314 raft_consensus.cc:2804] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.299336 17174 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:42.299784 16918 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.301146 17219 ts_tablet_manager.cc:1434] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:03:42.311863 17219 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:42.313477 17302 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.314008 17302 raft_consensus.cc:697] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:42.314828 17302 consensus_queue.cc:237] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.313031 17314 raft_consensus.cc:697] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:42.318130 17314 consensus_queue.cc:237] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.322455 17219 tablet_bootstrap.cc:654] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.355612 16848 catalog_manager.cc:5697] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.356786 17238 tablet_bootstrap.cc:654] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.363646 17338 raft_consensus.cc:493] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.364090 17219 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:42.364636 17219 ts_tablet_manager.cc:1403] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.016s
I20260709 13:03:42.364389 17338 raft_consensus.cc:515] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.366889 17338 leader_election.cc:290] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.367362 17219 raft_consensus.cc:359] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.367722 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.368108 17219 raft_consensus.cc:385] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.368422 17219 raft_consensus.cc:740] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.369093 17219 consensus_queue.cc:260] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:42.369972 17066 leader_election.cc:343] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.370180 17314 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.371461 17314 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.372762 17219 ts_tablet_manager.cc:1434] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:42.373559 17314 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.373801 17219 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:42.379880 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.380821 17024 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:42.382182 17219 tablet_bootstrap.cc:654] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.384902 17238 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:42.385432 17238 ts_tablet_manager.cc:1403] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.000s
I20260709 13:03:42.388659 17238 raft_consensus.cc:359] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.389606 17238 raft_consensus.cc:385] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.389958 17238 raft_consensus.cc:740] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.398411 16850 catalog_manager.cc:5697] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.401803 17238 consensus_queue.cc:260] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.408190 17356 raft_consensus.cc:493] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.406586 17339 raft_consensus.cc:493] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.408901 17356 raft_consensus.cc:515] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.409085 17339 raft_consensus.cc:515] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.412052 17339 leader_election.cc:290] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.405352 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.412849 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.413121 17025 raft_consensus.cc:2468] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:42.414014 17099 raft_consensus.cc:2393] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 1: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:42.417472 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.418097 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:42.420342 16918 leader_election.cc:343] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:42.421794 17067 leader_election.cc:343] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.422861 17067 leader_election.cc:304] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:42.425206 16918 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.426501 17314 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.426884 17314 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.427232 17314 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.438469 17238 ts_tablet_manager.cc:1434] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.011s
I20260709 13:03:42.438235 17339 raft_consensus.cc:2804] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.439200 17339 raft_consensus.cc:493] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.439666 17339 raft_consensus.cc:3060] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.442655 17356 leader_election.cc:290] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.443622 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bafbaa58b13148c2b916fe7ee7e234e6" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.446427 16993 leader_election.cc:304] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.446913 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.448042 17356 raft_consensus.cc:3060] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.452394 17238 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
W20260709 13:03:42.457312 17067 leader_election.cc:343] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.458467 17067 leader_election.cc:304] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.460662 17338 raft_consensus.cc:2749] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.463230 17338 raft_consensus.cc:493] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.463863 17338 raft_consensus.cc:515] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.467882 17338 leader_election.cc:290] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.468724 17238 tablet_bootstrap.cc:654] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.471823 17356 raft_consensus.cc:2749] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.520143 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c68d18d16941a9b3197081006477dc" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.524151 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c68d18d16941a9b3197081006477dc" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.525132 17025 raft_consensus.cc:2468] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:42.546663 16848 catalog_manager.cc:5697] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:42.549271 16848 catalog_manager.cc:5697] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.567042 16848 catalog_manager.cc:5697] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.598922 17218 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:42.599753 17218 ts_tablet_manager.cc:1403] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.343s	user 0.017s	sys 0.005s
I20260709 13:03:42.577378 17338 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.603859 17338 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.604480 17263 consensus_queue.cc:1048] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:42.522354 17067 leader_election.cc:343] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:42.607421 17067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 85116.9us
I20260709 13:03:42.609498 17067 leader_election.cc:304] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.613159 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.612669 17338 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:42.615247 17066 leader_election.cc:343] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.592576 17314 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.619014 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.622505 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:42.624413 17218 raft_consensus.cc:359] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:42.624897 16918 leader_election.cc:343] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.669566 17218 raft_consensus.cc:385] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:03:42.669723 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 44899.2us
I20260709 13:03:42.669955 17218 raft_consensus.cc:740] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
W20260709 13:03:42.625825 17067 leader_election.cc:343] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.670753 17339 raft_consensus.cc:515] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.671394 17067 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.672250 17338 raft_consensus.cc:2749] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:42.672392 17067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 50333.7us
I20260709 13:03:42.673763 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:42.638559 17343 consensus_queue.cc:1048] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:42.675268 17066 leader_election.cc:343] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.676091 17218 consensus_queue.cc:260] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.640765 17341 raft_consensus.cc:2804] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.678623 17341 raft_consensus.cc:493] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.678773 17220 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:42.679008 17341 raft_consensus.cc:3060] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.679311 17220 ts_tablet_manager.cc:1403] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.433s	user 0.136s	sys 0.169s
I20260709 13:03:42.682313 17220 raft_consensus.cc:359] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.682956 17220 raft_consensus.cc:385] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.683190 17218 ts_tablet_manager.cc:1434] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.083s	user 0.009s	sys 0.000s
I20260709 13:03:42.683218 17220 raft_consensus.cc:740] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.690745 17220 consensus_queue.cc:260] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.692081 17220 ts_tablet_manager.cc:1434] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:03:42.693069 17220 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:42.700052 17220 tablet_bootstrap.cc:654] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.701869 17219 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:42.702430 17219 ts_tablet_manager.cc:1403] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.329s	user 0.052s	sys 0.180s
I20260709 13:03:42.702446 17343 consensus_queue.cc:1048] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.705258 17219 raft_consensus.cc:359] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.708590 17219 raft_consensus.cc:385] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.708719 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.707803 17339 leader_election.cc:290] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.709527 17024 raft_consensus.cc:3060] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.710462 17218 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:42.713426 17302 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.713967 17302 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.716146 17302 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:42.717697 17218 tablet_bootstrap.cc:654] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.719648 17024 raft_consensus.cc:2468] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:42.708892 17219 raft_consensus.cc:740] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.720726 17067 leader_election.cc:304] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:42.721195 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.722276 17339 raft_consensus.cc:2804] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.722422 17263 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.722831 17339 raft_consensus.cc:697] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:42.723217 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.723865 17098 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:42.723758 17339 consensus_queue.cc:237] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.727669 16918 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:42.723017 17263 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:42.731447 17140 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:42.732694 17219 consensus_queue.cc:260] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.734550 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:42.736102 17140 leader_election.cc:343] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.738231 17237 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:42.738698 17237 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.739039 17237 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.747165 17237 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.749099 17356 consensus_queue.cc:1048] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.749255 17237 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:42.750006 17246 consensus_queue.cc:1048] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
W20260709 13:03:42.767199 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:42.768316 17341 raft_consensus.cc:515] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.752530 17219 ts_tablet_manager.cc:1434] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:03:42.754103 17220 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:42.780556 17220 ts_tablet_manager.cc:1403] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.004s
I20260709 13:03:42.782178 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c68d18d16941a9b3197081006477dc" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.783001 17025 raft_consensus.cc:3060] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.783350 17220 raft_consensus.cc:359] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.783998 17220 raft_consensus.cc:385] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.784252 17220 raft_consensus.cc:740] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.760309 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.785991 17219 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
W20260709 13:03:42.789654 17141 leader_election.cc:343] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.790133 17141 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.756187 17263 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:03:42.751101 16918 leader_election.cc:343] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.793396 17318 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.795377 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:42.784821 17220 consensus_queue.cc:260] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.797544 17237 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.798240 17220 ts_tablet_manager.cc:1434] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:03:42.799608 17220 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:42.800575 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:42.802142 16918 leader_election.cc:343] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.802942 17219 tablet_bootstrap.cc:654] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.804284 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:42.805058 17100 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.812184 17220 tablet_bootstrap.cc:654] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.812477 17024 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.813427 17237 consensus_queue.cc:1048] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:42.823288 17024 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.830704 16918 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.833590 17219 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:42.834185 17219 ts_tablet_manager.cc:1403] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.002s
I20260709 13:03:42.837779 17219 raft_consensus.cc:359] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.842427 17219 raft_consensus.cc:385] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.842792 17219 raft_consensus.cc:740] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
W20260709 13:03:42.843935 16918 consensus_peers.cc:597] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c -> Peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Couldn't send request to peer 8267993b082f449891e464bc4dee65f7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:42.841647 17433 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.842090 17220 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:42.844964 17220 ts_tablet_manager.cc:1403] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.001s
I20260709 13:03:42.848119 17220 raft_consensus.cc:359] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.843706 17219 consensus_queue.cc:260] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.849032 17433 raft_consensus.cc:697] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:42.850029 17433 consensus_queue.cc:237] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.851217 17219 ts_tablet_manager.cc:1434] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:03:42.854195 17219 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:42.855139 17100 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:42.854499 17220 raft_consensus.cc:385] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.857017 17220 raft_consensus.cc:740] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.856817 16918 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.858973 17319 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.859517 17319 raft_consensus.cc:697] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:42.860374 17319 consensus_queue.cc:237] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.858095 17220 consensus_queue.cc:260] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.874680 17343 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.874742 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c68d18d16941a9b3197081006477dc" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:42.875267 17343 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:42.876055 17067 leader_election.cc:343] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.775646 17314 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.879158 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.881520 17343 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.882485 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.883435 17024 raft_consensus.cc:2393] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 1: Already voted for candidate 64734f14219348f59d1cf8700416565c in this term.
I20260709 13:03:42.874541 17220 ts_tablet_manager.cc:1434] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:03:42.884792 17220 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:42.886413 17141 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.894243 17343 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.887599 17219 tablet_bootstrap.cc:654] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.901837 17343 raft_consensus.cc:2749] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.902711 17220 tablet_bootstrap.cc:654] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.920347 17238 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:42.920981 17238 ts_tablet_manager.cc:1403] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.469s	user 0.068s	sys 0.079s
I20260709 13:03:42.924824 17238 raft_consensus.cc:359] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.928200 17238 raft_consensus.cc:385] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.928501 17238 raft_consensus.cc:740] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.929157 17238 consensus_queue.cc:260] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.911401 17025 raft_consensus.cc:2468] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:42.934779 17433 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.935380 17433 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.937614 17433 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.941157 17238 ts_tablet_manager.cc:1434] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:03:42.942270 17238 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:42.944120 17343 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.944622 17343 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:42.946810 17343 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:42.948356 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:42.949150 17024 raft_consensus.cc:2393] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 1: Already voted for candidate 64734f14219348f59d1cf8700416565c in this term.
I20260709 13:03:42.951565 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:42.909569 17341 leader_election.cc:290] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:42.957680 17140 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:42.958559 17343 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:42.960332 17067 leader_election.cc:304] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.965586 17433 consensus_queue.cc:1048] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:42.967060 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:42.967255 17341 raft_consensus.cc:2804] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:42.968199 17341 raft_consensus.cc:697] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:42.975092 17218 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:42.976583 17218 ts_tablet_manager.cc:1403] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.293s	user 0.046s	sys 0.052s
I20260709 13:03:42.978637 17220 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:42.979524 17220 ts_tablet_manager.cc:1403] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.029s
I20260709 13:03:42.980201 17238 tablet_bootstrap.cc:654] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.980674 17218 raft_consensus.cc:359] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.981217 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:42.981570 17218 raft_consensus.cc:385] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.981990 17218 raft_consensus.cc:740] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.982882 17218 consensus_queue.cc:260] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:42.985383 17218 ts_tablet_manager.cc:1434] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 13:03:42.985443 16918 leader_election.cc:343] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.979707 17341 consensus_queue.cc:237] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:42.986541 17218 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:42.988240 17433 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:42.990023 16918 leader_election.cc:343] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:42.990531 16918 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:42.990124 17343 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.991581 17433 raft_consensus.cc:2749] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:42.991852 17219 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:42.992393 17219 ts_tablet_manager.cc:1403] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.140s	user 0.025s	sys 0.011s
I20260709 13:03:42.992417 17220 raft_consensus.cc:359] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.993251 17220 raft_consensus.cc:385] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.993541 17220 raft_consensus.cc:740] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:42.994238 17218 tablet_bootstrap.cc:654] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:42.994208 17220 consensus_queue.cc:260] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.996428 17219 raft_consensus.cc:359] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:42.997156 17219 raft_consensus.cc:385] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:42.997618 17220 ts_tablet_manager.cc:1434] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:03:42.997530 17219 raft_consensus.cc:740] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.003989 17219 consensus_queue.cc:260] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.006322 17219 ts_tablet_manager.cc:1434] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:43.006459 16846 catalog_manager.cc:5697] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.007431 17219 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:42.998793 17220 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:43.021252 17435 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:03:43.032275 17027 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.033156 17220 tablet_bootstrap.cc:654] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.041532 17238 tablet_bootstrap.cc:492] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:43.042044 17238 ts_tablet_manager.cc:1403] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.100s	user 0.013s	sys 0.001s
I20260709 13:03:43.042989 17219 tablet_bootstrap.cc:654] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.037776 17341 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.053815 17218 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:43.055734 17218 ts_tablet_manager.cc:1403] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.006s
I20260709 13:03:43.053987 17341 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.054970 16846 catalog_manager.cc:5697] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.050980 17263 raft_consensus.cc:493] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.044760 17238 raft_consensus.cc:359] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.057711 17238 raft_consensus.cc:385] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.057952 17238 raft_consensus.cc:740] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.059229 17238 consensus_queue.cc:260] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.061460 17263 raft_consensus.cc:515] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.063939 17263 leader_election.cc:290] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:43.064990 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:43.065673 17025 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:43.066941 17238 ts_tablet_manager.cc:1434] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:03:43.067420 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.067951 17238 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:43.068881 17141 leader_election.cc:304] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:43.069880 17263 raft_consensus.cc:2804] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.070354 17263 raft_consensus.cc:493] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.070755 17263 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.068032 17099 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:43.073064 17218 raft_consensus.cc:359] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.073792 17218 raft_consensus.cc:385] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.074870 17220 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:43.075379 17220 ts_tablet_manager.cc:1403] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.001s
I20260709 13:03:43.079411 17238 tablet_bootstrap.cc:654] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.080392 17220 raft_consensus.cc:359] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.081574 17341 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:43.083564 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:43.084549 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:43.074061 17218 raft_consensus.cc:740] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
W20260709 13:03:43.085510 17067 leader_election.cc:343] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.086369 17218 consensus_queue.cc:260] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.089159 17218 ts_tablet_manager.cc:1434] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:03:43.090478 17220 raft_consensus.cc:385] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.091267 17220 raft_consensus.cc:740] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.091758 17218 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
W20260709 13:03:43.092777 17067 leader_election.cc:343] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.092684 17220 consensus_queue.cc:260] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.093470 17067 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:43.095173 17341 raft_consensus.cc:2749] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.096961 17220 ts_tablet_manager.cc:1434] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:03:43.098057 17220 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:43.098583 16848 catalog_manager.cc:5697] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.111302 16848 catalog_manager.cc:5697] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.115485 17263 raft_consensus.cc:515] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.119993 17218 tablet_bootstrap.cc:654] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.124521 16952 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.130788 17237 raft_consensus.cc:493] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.135897 17237 raft_consensus.cc:515] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.138089 17237 leader_election.cc:290] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:43.139101 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.139520 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f8e87277d714ded843585986eae389e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:43.140336 17024 raft_consensus.cc:2393] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:43.141896 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:43.142347 16918 leader_election.cc:304] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:43.142506 17099 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.145782 17433 raft_consensus.cc:3060] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.146860 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:43.147584 17024 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.148212 17219 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:43.148859 17219 ts_tablet_manager.cc:1403] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.142s	user 0.017s	sys 0.022s
I20260709 13:03:43.149719 17220 tablet_bootstrap.cc:654] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.152783 17219 raft_consensus.cc:359] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.154419 17219 raft_consensus.cc:385] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.154700 17219 raft_consensus.cc:740] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.156509 17219 consensus_queue.cc:260] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.139474 17263 leader_election.cc:290] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:43.172804 17238 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:43.173400 17238 ts_tablet_manager.cc:1403] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.106s	user 0.022s	sys 0.003s
I20260709 13:03:43.178167 17219 ts_tablet_manager.cc:1434] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 13:03:43.179492 17219 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:43.185595 17219 tablet_bootstrap.cc:654] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.184326 17099 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:43.193569 17024 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:43.194814 17141 leader_election.cc:304] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:43.195413 17218 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:43.195912 17218 ts_tablet_manager.cc:1403] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.004s
I20260709 13:03:43.198011 17318 raft_consensus.cc:2804] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.198426 17433 raft_consensus.cc:2749] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.198594 17318 raft_consensus.cc:697] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:43.199100 17218 raft_consensus.cc:359] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.199837 17218 raft_consensus.cc:385] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.199587 17318 consensus_queue.cc:237] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.200183 17218 raft_consensus.cc:740] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.200933 17218 consensus_queue.cc:260] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.203910 17218 ts_tablet_manager.cc:1434] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:43.205051 17218 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:43.222831 17238 raft_consensus.cc:359] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:43.223621 17238 raft_consensus.cc:385] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.223887 17238 raft_consensus.cc:740] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.224524 17238 consensus_queue.cc:260] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:43.229642 17238 ts_tablet_manager.cc:1434] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.056s	user 0.002s	sys 0.005s
I20260709 13:03:43.230650 17238 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:43.240160 17263 raft_consensus.cc:493] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.240763 17263 raft_consensus.cc:515] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.242501 17433 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.244983 17263 leader_election.cc:290] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.246213 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.253654 17238 tablet_bootstrap.cc:654] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.265463 17220 tablet_bootstrap.cc:492] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:43.265986 17220 ts_tablet_manager.cc:1403] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.000s
I20260709 13:03:43.268419 17218 tablet_bootstrap.cc:654] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.257481 17099 raft_consensus.cc:2468] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:43.272615 17141 leader_election.cc:304] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:43.278702 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:43.283511 17140 leader_election.cc:343] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.274749 17318 raft_consensus.cc:2804] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.286398 17318 raft_consensus.cc:493] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.286768 17318 raft_consensus.cc:3060] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.285439 17220 raft_consensus.cc:359] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.288276 17220 raft_consensus.cc:385] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.288661 17220 raft_consensus.cc:740] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.290606 17220 consensus_queue.cc:260] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:43.304373 17341 consensus_queue.cc:1048] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.307833 17220 ts_tablet_manager.cc:1434] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:03:43.308713 17220 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:43.313153 17237 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.317003 16848 catalog_manager.cc:5697] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.322149 17319 consensus_queue.cc:1048] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.328464 17341 consensus_queue.cc:1048] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:43.341975 16918 consensus_peers.cc:597] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:43.363773 17219 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:43.364302 17219 ts_tablet_manager.cc:1403] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.185s	user 0.028s	sys 0.013s
I20260709 13:03:43.340807 17318 raft_consensus.cc:515] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.372025 17318 leader_election.cc:290] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.373860 17218 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:43.374394 17218 ts_tablet_manager.cc:1403] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.170s	user 0.018s	sys 0.005s
I20260709 13:03:43.375039 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:43.375684 17098 raft_consensus.cc:3060] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.376691 17238 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:43.377185 17238 ts_tablet_manager.cc:1403] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.147s	user 0.028s	sys 0.042s
I20260709 13:03:43.377045 17341 raft_consensus.cc:493] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.381136 17341 raft_consensus.cc:515] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.383277 17341 leader_election.cc:290] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:43.391634 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.385471 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:43.393769 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:43.394917 17067 leader_election.cc:343] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.397923 17218 raft_consensus.cc:359] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.398484 17218 raft_consensus.cc:385] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.398710 17218 raft_consensus.cc:740] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.399210 17218 consensus_queue.cc:260] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:43.401803 17066 leader_election.cc:343] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.402285 17066 leader_election.cc:304] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:43.403614 17339 raft_consensus.cc:2749] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:43.409972 17140 leader_election.cc:343] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.412204 17218 ts_tablet_manager.cc:1434] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:03:43.412385 17219 raft_consensus.cc:359] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.413142 17219 raft_consensus.cc:385] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.413528 17219 raft_consensus.cc:740] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.414438 17219 consensus_queue.cc:260] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.416822 17219 ts_tablet_manager.cc:1434] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:03:43.413233 17218 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:43.419592 17220 tablet_bootstrap.cc:654] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.421132 17219 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:43.421422 17238 raft_consensus.cc:359] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.424006 17238 raft_consensus.cc:385] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.426365 17238 raft_consensus.cc:740] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.427078 17238 consensus_queue.cc:260] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.440480 17238 ts_tablet_manager.cc:1434] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:03:43.444890 17460 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.445668 17460 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.446674 17219 tablet_bootstrap.cc:654] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.449231 17460 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.453756 17238 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:43.452996 17174 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.458143 17098 raft_consensus.cc:2468] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:43.461994 17141 leader_election.cc:304] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:43.463197 17263 raft_consensus.cc:2804] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.463680 17263 raft_consensus.cc:697] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:43.464543 17263 consensus_queue.cc:237] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.475953 17341 consensus_queue.cc:1048] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.477180 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.450510 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.484047 17218 tablet_bootstrap.cc:654] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.499210 17220 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:43.500396 17220 ts_tablet_manager.cc:1403] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.192s	user 0.034s	sys 0.013s
I20260709 13:03:43.513116 17220 raft_consensus.cc:359] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.515400 17220 raft_consensus.cc:385] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.515847 17220 raft_consensus.cc:740] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.516781 17220 consensus_queue.cc:260] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.524376 17220 ts_tablet_manager.cc:1434] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:03:43.521790 17246 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.526269 17246 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.528412 17246 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:43.531001 17140 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:43.532316 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:43.534907 16993 leader_election.cc:343] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.525368 17220 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:43.521548 17466 raft_consensus.cc:493] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.539345 17471 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.535420 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.539885 17471 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.544029 17341 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.541621 17466 raft_consensus.cc:515] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.544621 17341 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:43.541554 16992 leader_election.cc:343] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.546063 16992 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:43.546802 17467 raft_consensus.cc:2749] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.547668 17466 leader_election.cc:290] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.546999 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd44ab20c334b99964d344244e650ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:43.550554 17471 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.552120 17238 tablet_bootstrap.cc:654] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:43.551985 17067 leader_election.cc:343] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.559422 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd44ab20c334b99964d344244e650ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.560321 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:43.560525 16952 raft_consensus.cc:2468] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
W20260709 13:03:43.561880 17067 leader_election.cc:343] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.567219 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.567868 16950 raft_consensus.cc:2468] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 0.
I20260709 13:03:43.567221 17219 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:43.569047 17219 ts_tablet_manager.cc:1403] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.148s	user 0.013s	sys 0.012s
I20260709 13:03:43.578487 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.580348 17219 raft_consensus.cc:359] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.581053 17219 raft_consensus.cc:385] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.570868 17066 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.582523 17066 leader_election.cc:304] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.583917 17466 raft_consensus.cc:2804] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.584424 17466 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.584864 17466 raft_consensus.cc:3060] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.586014 17341 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:43.586637 17341 raft_consensus.cc:2804] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.586918 17341 raft_consensus.cc:493] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.587212 17341 raft_consensus.cc:3060] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.581372 17219 raft_consensus.cc:740] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.591233 17219 consensus_queue.cc:260] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.593421 17219 ts_tablet_manager.cc:1434] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:03:43.593633 17467 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.594297 17467 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:43.595918 17175 raft_consensus.cc:3060] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.598151 17467 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.594925 17219 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:43.608007 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:43.610467 16993 leader_election.cc:343] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.614121 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.614711 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:43.616178 17067 leader_election.cc:343] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.617592 17460 raft_consensus.cc:2749] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:43.616261 16992 leader_election.cc:343] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.618897 16992 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.619557 17467 raft_consensus.cc:2749] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:43.620131 17066 leader_election.cc:343] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.620627 17066 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:43.621578 17470 raft_consensus.cc:2749] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.611888 17319 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.622129 17319 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.624295 17319 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:43.628528 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:43.629206 17172 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:43.629580 17466 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.630522 16918 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:43.632048 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.632750 17433 raft_consensus.cc:2804] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:43.632759 17098 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:43.633375 17433 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.633836 17433 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.635720 17341 raft_consensus.cc:515] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.636396 17218 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:43.636893 17218 ts_tablet_manager.cc:1403] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.224s	user 0.023s	sys 0.006s
I20260709 13:03:43.639652 17341 leader_election.cc:290] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.639721 17471 consensus_queue.cc:1048] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.641007 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd44ab20c334b99964d344244e650ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:43.642445 16951 raft_consensus.cc:3060] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.643232 16848 catalog_manager.cc:5697] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.651437 17220 tablet_bootstrap.cc:654] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.654875 17218 raft_consensus.cc:359] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.663458 17218 raft_consensus.cc:385] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.663733 17218 raft_consensus.cc:740] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.664390 17218 consensus_queue.cc:260] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.666460 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:43.666983 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:43.667318 16952 raft_consensus.cc:3060] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:43.668388 17067 leader_election.cc:343] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.653757 17219 tablet_bootstrap.cc:654] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.655400 16951 raft_consensus.cc:2468] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:43.662688 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd44ab20c334b99964d344244e650ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:43.656168 17433 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.680008 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:43.680588 17098 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.683017 17433 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:43.684984 17218 ts_tablet_manager.cc:1434] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 13:03:43.686208 17218 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:43.674880 17066 leader_election.cc:304] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
W20260709 13:03:43.688740 17067 leader_election.cc:343] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.688952 17341 raft_consensus.cc:2804] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.689586 17341 raft_consensus.cc:697] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:43.690400 17341 consensus_queue.cc:237] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.693653 16952 raft_consensus.cc:2468] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:43.651005 17466 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.696837 17462 consensus_queue.cc:1048] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:43.697516 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:43.705547 17238 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:43.706553 17238 ts_tablet_manager.cc:1403] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.265s	user 0.011s	sys 0.004s
I20260709 13:03:43.709609 17238 raft_consensus.cc:359] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.710592 17238 raft_consensus.cc:385] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.710996 17238 raft_consensus.cc:740] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.704213 17098 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:43.710263 17066 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.714485 17466 raft_consensus.cc:2804] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.714625 16918 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:43.715404 17466 raft_consensus.cc:697] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:43.716089 17433 raft_consensus.cc:2804] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:43.716727 17433 raft_consensus.cc:697] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:43.717615 17433 consensus_queue.cc:237] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.712598 17238 consensus_queue.cc:260] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.720531 17462 consensus_queue.cc:1048] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:43.720767 17174 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.732687 17238 ts_tablet_manager.cc:1434] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:43.733708 17238 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:43.739629 17238 tablet_bootstrap.cc:654] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.744014 17220 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:43.744416 17220 ts_tablet_manager.cc:1403] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.219s	user 0.027s	sys 0.009s
I20260709 13:03:43.716563 17466 consensus_queue.cc:237] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.748451 17219 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:43.748620 17218 tablet_bootstrap.cc:654] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.748992 17219 ts_tablet_manager.cc:1403] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.154s	user 0.017s	sys 0.000s
I20260709 13:03:43.755198 16848 catalog_manager.cc:5697] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.758315 17219 raft_consensus.cc:359] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.759032 17219 raft_consensus.cc:385] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.759336 17219 raft_consensus.cc:740] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.762708 17219 consensus_queue.cc:260] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.765738 17220 raft_consensus.cc:359] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.766450 17220 raft_consensus.cc:385] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.766691 17220 raft_consensus.cc:740] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.767270 17220 consensus_queue.cc:260] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.776660 17219 ts_tablet_manager.cc:1434] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:03:43.780011 17219 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:43.777066 17302 raft_consensus.cc:493] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.784351 17302 raft_consensus.cc:515] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.769220 17220 ts_tablet_manager.cc:1434] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.000s
I20260709 13:03:43.786285 17220 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:43.786422 17302 leader_election.cc:290] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:43.791885 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:43.792852 17099 raft_consensus.cc:2393] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:43.793854 17219 tablet_bootstrap.cc:654] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.795713 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed56f28edb0c443ead4f92414e0a29fa" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:43.763810 17174 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:43.803171 16918 leader_election.cc:304] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.804557 17302 raft_consensus.cc:3060] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.809923 17220 tablet_bootstrap.cc:654] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.833420 17218 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:43.834004 17218 ts_tablet_manager.cc:1403] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.148s	user 0.018s	sys 0.005s
I20260709 13:03:43.845463 17027 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:43.848254 16846 catalog_manager.cc:5697] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.851675 17238 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:43.852217 17238 ts_tablet_manager.cc:1403] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.005s
I20260709 13:03:43.857264 17238 raft_consensus.cc:359] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.858388 17238 raft_consensus.cc:385] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.858832 17238 raft_consensus.cc:740] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.859912 17238 consensus_queue.cc:260] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.863222 16847 catalog_manager.cc:5697] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.864347 17238 ts_tablet_manager.cc:1434] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:03:43.865548 17238 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:43.870852 17218 raft_consensus.cc:359] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.872139 17218 raft_consensus.cc:385] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.872392 17218 raft_consensus.cc:740] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.920339 17466 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.923909 17238 tablet_bootstrap.cc:654] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:43.920905 17466 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:43.927151 17466 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:43.870496 17302 raft_consensus.cc:2749] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.928225 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:43.929320 17218 consensus_queue.cc:260] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.930596 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:43.930864 17066 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.932611 17218 ts_tablet_manager.cc:1434] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.000s
W20260709 13:03:43.933092 17067 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:43.933632 17218 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:43.933765 17067 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:43.937467 17219 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:43.937960 17219 ts_tablet_manager.cc:1403] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.158s	user 0.020s	sys 0.015s
I20260709 13:03:43.941104 17219 raft_consensus.cc:359] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.941797 17219 raft_consensus.cc:385] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:43.942054 17219 raft_consensus.cc:740] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:43.942790 17460 consensus_queue.cc:1048] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:43.947300 17219 consensus_queue.cc:260] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:43.944608 17466 raft_consensus.cc:2749] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:43.967614 17218 tablet_bootstrap.cc:654] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.000229 17219 ts_tablet_manager.cc:1434] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 13:03:44.001307 17219 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.015779 17460 consensus_queue.cc:1048] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.021948 17238 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:44.022555 17238 ts_tablet_manager.cc:1403] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.157s	user 0.010s	sys 0.024s
I20260709 13:03:44.028985 17220 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.029767 17220 ts_tablet_manager.cc:1403] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.244s	user 0.029s	sys 0.026s
I20260709 13:03:44.034022 17220 raft_consensus.cc:359] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.035041 17220 raft_consensus.cc:385] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.035442 17220 raft_consensus.cc:740] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.038493 17220 consensus_queue.cc:260] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.043150 17238 raft_consensus.cc:359] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.043872 17238 raft_consensus.cc:385] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.044150 17238 raft_consensus.cc:740] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.047015 17318 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.047554 17318 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.049618 17318 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:44.050521 17220 ts_tablet_manager.cc:1434] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:03:44.051520 17220 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.052467 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.044773 17238 consensus_queue.cc:260] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.055015 17238 ts_tablet_manager.cc:1434] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:03:44.056243 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162f7d6234e549cc968efac4ee8846ac" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.056705 17238 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:44.056285 16951 raft_consensus.cc:2393] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 1: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:44.059712 17219 tablet_bootstrap.cc:654] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.062301 17140 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:44.063555 17218 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.064040 17218 ts_tablet_manager.cc:1403] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.131s	user 0.018s	sys 0.014s
I20260709 13:03:44.065192 17263 raft_consensus.cc:3060] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.079811 17263 raft_consensus.cc:2749] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.083907 17220 tablet_bootstrap.cc:654] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.096659 17238 tablet_bootstrap.cc:654] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.098757 17218 raft_consensus.cc:359] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.100394 17218 raft_consensus.cc:385] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.100755 17218 raft_consensus.cc:740] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.101656 17218 consensus_queue.cc:260] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.104513 17218 ts_tablet_manager.cc:1434] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:03:44.105826 17218 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.111562 17433 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.112082 17433 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.113590 17218 tablet_bootstrap.cc:654] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.116035 17433 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.117439 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.119297 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.119707 17024 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.120210 17100 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.121086 16918 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
W20260709 13:03:44.138934 17067 consensus_peers.cc:597] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Couldn't send request to peer 8267993b082f449891e464bc4dee65f7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:44.139072 17319 raft_consensus.cc:2804] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.139968 17319 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.140573 17319 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.146994 17218 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.147537 17218 ts_tablet_manager.cc:1403] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.009s
W20260709 13:03:44.152415 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:44.155730 17218 raft_consensus.cc:359] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.157765 17218 raft_consensus.cc:385] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.158099 17218 raft_consensus.cc:740] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.158923 17219 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:44.159490 17219 ts_tablet_manager.cc:1403] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.158s	user 0.032s	sys 0.042s
I20260709 13:03:44.164064 17433 consensus_queue.cc:1048] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.158949 17218 consensus_queue.cc:260] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.166829 17218 ts_tablet_manager.cc:1434] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.002s
I20260709 13:03:44.167762 17218 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.168562 17219 raft_consensus.cc:359] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.169262 17219 raft_consensus.cc:385] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.169571 17219 raft_consensus.cc:740] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.170300 17219 consensus_queue.cc:260] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.172386 17219 ts_tablet_manager.cc:1434] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.004s
I20260709 13:03:44.173226 17238 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:44.175071 17238 ts_tablet_manager.cc:1403] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.003s
I20260709 13:03:44.157061 17319 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.173374 17219 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.179965 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.180279 17218 tablet_bootstrap.cc:654] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.180555 17024 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.187577 17219 tablet_bootstrap.cc:654] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.198581 17024 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.199951 16918 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:44.200312 17341 consensus_queue.cc:1048] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.200672 17302 raft_consensus.cc:2804] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.201234 17302 raft_consensus.cc:697] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:44.202060 17302 consensus_queue.cc:237] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.209555 17220 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.209947 17219 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:44.211602 17219 ts_tablet_manager.cc:1403] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.003s
I20260709 13:03:44.211341 17341 consensus_queue.cc:1048] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.214558 17219 raft_consensus.cc:359] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.215279 17219 raft_consensus.cc:385] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.215525 17219 raft_consensus.cc:740] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.216104 17219 consensus_queue.cc:260] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.217646 17220 ts_tablet_manager.cc:1403] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.166s	user 0.007s	sys 0.007s
I20260709 13:03:44.221233 17341 consensus_queue.cc:1048] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:44.221661 17220 raft_consensus.cc:359] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.222505 17220 raft_consensus.cc:385] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.222855 17438 consensus_queue.cc:1048] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.224089 17435 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.224632 17435 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.227221 17435 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.229053 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.229229 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.229812 17024 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.229969 17100 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.231019 16918 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:44.232337 17433 raft_consensus.cc:2804] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.232863 17433 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.233276 17433 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.238039 17319 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.240633 17238 raft_consensus.cc:359] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.241379 17238 raft_consensus.cc:385] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.241642 17238 raft_consensus.cc:740] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.244199 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.245051 17098 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.247704 17433 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.253201 17219 ts_tablet_manager.cc:1434] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:03:44.253232 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.254076 17024 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.254441 17219 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.257567 17238 consensus_queue.cc:260] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.259207 17238 ts_tablet_manager.cc:1434] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.083s	user 0.003s	sys 0.001s
I20260709 13:03:44.260223 17238 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:44.222777 17220 raft_consensus.cc:740] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.262123 17220 consensus_queue.cc:260] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.265175 17220 ts_tablet_manager.cc:1434] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.001s
I20260709 13:03:44.266119 17220 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.268536 17098 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.273485 17024 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.277068 17262 raft_consensus.cc:493] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.292663 17218 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.292462 17262 raft_consensus.cc:515] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.293171 17218 ts_tablet_manager.cc:1403] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.009s
I20260709 13:03:44.294766 17262 leader_election.cc:290] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.278345 16848 catalog_manager.cc:5697] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.295928 17218 raft_consensus.cc:359] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.296761 17218 raft_consensus.cc:385] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.279618 17467 raft_consensus.cc:493] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.297498 17218 raft_consensus.cc:740] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.297791 17467 raft_consensus.cc:515] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.298220 17218 consensus_queue.cc:260] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.299887 17467 leader_election.cc:290] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:44.300428 17218 ts_tablet_manager.cc:1434] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:44.301590 17218 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.302645 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.303095 17263 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.303589 17263 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.282729 17219 tablet_bootstrap.cc:654] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.305681 17263 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.284349 17433 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:03:44.306969 16992 leader_election.cc:343] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.284544 17238 tablet_bootstrap.cc:654] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.308473 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.309203 16950 raft_consensus.cc:2468] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.287562 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.310535 17140 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.310837 17100 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.289562 17220 tablet_bootstrap.cc:654] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.312459 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.313011 17318 raft_consensus.cc:2804] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.313133 17099 raft_consensus.cc:2468] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:44.313472 17318 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.313903 17318 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.315063 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.315358 16993 leader_election.cc:304] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:44.317595 17467 raft_consensus.cc:2804] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.317989 17467 raft_consensus.cc:493] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.318308 17467 raft_consensus.cc:3060] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.318356 16918 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:44.317590 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.321313 17433 raft_consensus.cc:2804] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.321482 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:44.322099 17433 raft_consensus.cc:697] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:44.322302 17343 raft_consensus.cc:493] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.323000 17433 consensus_queue.cc:237] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:44.316996 16993 leader_election.cc:343] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:44.328995 17141 leader_election.cc:343] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.322764 17343 raft_consensus.cc:515] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.331272 17343 leader_election.cc:290] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.337714 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2db96a4ed139460aa021acdd63b9b529" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.338587 17098 raft_consensus.cc:2468] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.340039 17141 leader_election.cc:304] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.341209 17343 raft_consensus.cc:2804] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.341713 17343 raft_consensus.cc:493] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.342070 17343 raft_consensus.cc:3060] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.349821 17318 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.352970 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:44.353803 16950 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.352418 17467 raft_consensus.cc:515] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.361120 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:44.361399 17318 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.361660 16950 raft_consensus.cc:2468] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.366144 17218 tablet_bootstrap.cc:654] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.370757 17467 leader_election.cc:290] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:03:44.370132 16992 leader_election.cc:343] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.374058 17140 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.374960 17263 raft_consensus.cc:2804] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.375401 17263 raft_consensus.cc:697] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:44.376616 16846 catalog_manager.cc:5697] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.376112 17263 consensus_queue.cc:237] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.383457 17100 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.384884 17219 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:44.385727 17219 ts_tablet_manager.cc:1403] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.004s
I20260709 13:03:44.386159 17238 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:44.386736 17238 ts_tablet_manager.cc:1403] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.003s
I20260709 13:03:44.388602 17219 raft_consensus.cc:359] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.389348 17219 raft_consensus.cc:385] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.389600 17219 raft_consensus.cc:740] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.390245 17219 consensus_queue.cc:260] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:44.393841 16993 leader_election.cc:343] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.395345 17219 ts_tablet_manager.cc:1434] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:03:44.395579 17238 raft_consensus.cc:359] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.396358 17238 raft_consensus.cc:385] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.396714 17238 raft_consensus.cc:740] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.396377 17219 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.393043 17343 raft_consensus.cc:515] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.406584 17343 leader_election.cc:290] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.400516 17238 consensus_queue.cc:260] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.408921 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2db96a4ed139460aa021acdd63b9b529" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.409593 17098 raft_consensus.cc:3060] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.418570 17218 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.419085 17218 ts_tablet_manager.cc:1403] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.118s	user 0.005s	sys 0.009s
I20260709 13:03:44.419505 17219 tablet_bootstrap.cc:654] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.400264 16847 catalog_manager.cc:5697] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.399746 16993 leader_election.cc:304] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:44.424705 17511 raft_consensus.cc:2749] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.425741 17220 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.426245 17220 ts_tablet_manager.cc:1403] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.009s
I20260709 13:03:44.428862 17220 raft_consensus.cc:359] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.429523 17220 raft_consensus.cc:385] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.429800 17220 raft_consensus.cc:740] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.431742 17218 raft_consensus.cc:359] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.432731 17218 raft_consensus.cc:385] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.433008 17218 raft_consensus.cc:740] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.434125 17218 consensus_queue.cc:260] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.430424 17220 consensus_queue.cc:260] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.440609 17238 ts_tablet_manager.cc:1434] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.006s
I20260709 13:03:44.449692 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2db96a4ed139460aa021acdd63b9b529" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.451043 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.452158 17218 ts_tablet_manager.cc:1434] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
I20260709 13:03:44.452512 17238 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:44.455790 17218 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.456605 17220 ts_tablet_manager.cc:1434] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 13:03:44.459925 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.460892 17220 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.461484 17099 raft_consensus.cc:3060] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:44.462245 17141 leader_election.cc:343] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:44.463730 17141 leader_election.cc:343] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.464013 17098 raft_consensus.cc:2468] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.462186 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2db96a4ed139460aa021acdd63b9b529" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.465878 17141 leader_election.cc:304] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.473680 17343 raft_consensus.cc:2804] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.474445 17343 raft_consensus.cc:697] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:44.475374 17343 consensus_queue.cc:237] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:44.476737 17141 leader_election.cc:343] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.479566 17218 tablet_bootstrap.cc:654] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.483646 17099 raft_consensus.cc:2468] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:44.478541 17219 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:44.486915 17219 ts_tablet_manager.cc:1403] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.091s	user 0.023s	sys 0.023s
I20260709 13:03:44.486961 16993 leader_election.cc:304] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:44.488139 17511 raft_consensus.cc:2804] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.488617 17511 raft_consensus.cc:697] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:44.489482 17511 consensus_queue.cc:237] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.489976 17219 raft_consensus.cc:359] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.490667 17219 raft_consensus.cc:385] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.490944 17219 raft_consensus.cc:740] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.491585 17219 consensus_queue.cc:260] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.492120 17220 tablet_bootstrap.cc:654] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.493885 17219 ts_tablet_manager.cc:1434] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:03:44.495846 17219 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.501150 17238 tablet_bootstrap.cc:654] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.502377 17219 tablet_bootstrap.cc:654] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.504021 17341 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.504639 17341 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.506930 17341 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:44.509110 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:44.509781 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:44.511950 17067 leader_election.cc:343] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:44.512775 17067 leader_election.cc:343] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.512967 17433 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.513353 17067 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:44.514750 17341 raft_consensus.cc:2749] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.515774 17220 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.516336 17220 ts_tablet_manager.cc:1403] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.056s	user 0.007s	sys 0.004s
I20260709 13:03:44.517580 17433 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.518064 16845 catalog_manager.cc:5697] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.521085 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:44.522789 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.523474 17025 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.523963 17433 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.527189 16918 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
W20260709 13:03:44.528064 16918 leader_election.cc:343] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.538136 17218 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.538689 17238 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:44.538844 17218 ts_tablet_manager.cc:1403] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.001s
I20260709 13:03:44.539184 17238 ts_tablet_manager.cc:1403] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.000s
I20260709 13:03:44.542493 17218 raft_consensus.cc:359] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.543421 17218 raft_consensus.cc:385] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.543737 17218 raft_consensus.cc:740] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.544725 17341 raft_consensus.cc:493] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.545390 17341 raft_consensus.cc:515] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.546521 17433 raft_consensus.cc:2804] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.547119 17433 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.547534 17433 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.519773 17220 raft_consensus.cc:359] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.548942 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.549499 17220 raft_consensus.cc:385] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.549839 17220 raft_consensus.cc:740] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.550503 17220 consensus_queue.cc:260] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:44.550921 17066 leader_election.cc:343] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.552631 17220 ts_tablet_manager.cc:1434] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:03:44.553571 17220 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.553725 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:44.555411 17067 leader_election.cc:343] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.556044 17067 leader_election.cc:304] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:44.557052 17238 raft_consensus.cc:359] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.557408 17505 raft_consensus.cc:2749] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.557814 17238 raft_consensus.cc:385] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.558166 17238 raft_consensus.cc:740] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.558791 17341 leader_election.cc:290] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.558945 17238 consensus_queue.cc:260] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.560010 17218 consensus_queue.cc:260] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.563284 17218 ts_tablet_manager.cc:1434] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:03:44.566669 17238 ts_tablet_manager.cc:1434] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:03:44.569548 17218 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.569778 17238 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:44.576939 17218 tablet_bootstrap.cc:654] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.580492 17511 raft_consensus.cc:493] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.582012 17511 raft_consensus.cc:515] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.583485 17220 tablet_bootstrap.cc:654] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.586244 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "117045952bb14176abd739933fa51259" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.587631 16850 catalog_manager.cc:5697] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.589324 17219 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
W20260709 13:03:44.590072 16992 leader_election.cc:343] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.596438 17438 consensus_queue.cc:1048] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.598191 17219 ts_tablet_manager.cc:1403] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.027s
I20260709 13:03:44.599944 17433 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.631282 17343 raft_consensus.cc:493] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.602818 17511 leader_election.cc:290] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.603917 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "117045952bb14176abd739933fa51259" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.634891 17100 raft_consensus.cc:2468] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:44.610183 17238 tablet_bootstrap.cc:654] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.610946 17219 raft_consensus.cc:359] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.614797 17460 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.625317 17220 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.633103 17438 consensus_queue.cc:1048] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.633628 17343 raft_consensus.cc:515] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.636138 17219 raft_consensus.cc:385] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.637863 17219 raft_consensus.cc:740] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.638141 16993 leader_election.cc:304] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:44.638592 17219 consensus_queue.cc:260] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.639465 17511 raft_consensus.cc:2804] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.639863 17511 raft_consensus.cc:493] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.637023 17220 ts_tablet_manager.cc:1403] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.004s
I20260709 13:03:44.639932 17460 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.640270 17511 raft_consensus.cc:3060] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.642670 17460 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.643122 17220 raft_consensus.cc:359] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.643901 17220 raft_consensus.cc:385] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.645898 17220 raft_consensus.cc:740] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.646525 17220 consensus_queue.cc:260] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.650282 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.651000 17098 raft_consensus.cc:2468] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.651280 17343 leader_election.cc:290] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.655969 17220 ts_tablet_manager.cc:1434] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:44.656656 17219 ts_tablet_manager.cc:1434] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.002s
I20260709 13:03:44.652309 17341 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.643702 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.658023 17219 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.656656 17141 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.665942 17220 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.658435 17262 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.671001 17141 leader_election.cc:343] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.665247 17460 raft_consensus.cc:2804] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.669275 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ce161acdcd435a8f869ae50a6ea162" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.669157 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ce161acdcd435a8f869ae50a6ea162" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.674691 16950 raft_consensus.cc:2468] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.672176 17262 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:44.675565 17141 leader_election.cc:343] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.677196 17262 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:44.678120 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:44.678843 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.686770 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:44.687436 16993 leader_election.cc:343] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.675801 17460 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.687858 17027 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:44.688180 16992 leader_election.cc:343] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.688681 16992 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:44.687085 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:44.690301 17262 raft_consensus.cc:2749] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.692505 17433 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.686348 17140 leader_election.cc:304] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:44.662735 17341 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.695515 17318 raft_consensus.cc:2804] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:44.697209 16918 leader_election.cc:343] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.698045 17219 tablet_bootstrap.cc:654] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.700676 17318 raft_consensus.cc:493] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.702718 17318 raft_consensus.cc:3060] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.706219 17220 tablet_bootstrap.cc:654] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.688230 17460 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.732208 17027 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.735814 16918 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:44.736304 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:44.737064 17302 raft_consensus.cc:2804] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.737721 17302 raft_consensus.cc:697] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
W20260709 13:03:44.737946 17066 leader_election.cc:343] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.738693 17302 consensus_queue.cc:237] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.735824 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:44.742089 17067 leader_election.cc:343] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.742555 17067 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:44.733915 17341 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:44.743772 17466 raft_consensus.cc:2749] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:44.745151 17438 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.745698 17438 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.747776 17438 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:44.750545 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.751216 17099 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:44.752686 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:44.756108 16918 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
W20260709 13:03:44.757086 16918 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.758566 17438 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.759065 17438 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.758225 17218 tablet_bootstrap.cc:492] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.759804 17218 ts_tablet_manager.cc:1403] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.191s	user 0.025s	sys 0.018s
I20260709 13:03:44.763571 17218 raft_consensus.cc:359] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.763731 17511 raft_consensus.cc:515] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.765067 17218 raft_consensus.cc:385] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.765678 17218 raft_consensus.cc:740] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.766708 17438 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.769996 17218 consensus_queue.cc:260] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.784874 17238 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:44.785843 17238 ts_tablet_manager.cc:1403] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.218s	user 0.020s	sys 0.000s
I20260709 13:03:44.774606 17343 raft_consensus.cc:493] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.788412 17343 raft_consensus.cc:515] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.790644 17343 leader_election.cc:290] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.792645 17511 leader_election.cc:290] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.794939 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "117045952bb14176abd739933fa51259" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.795670 17100 raft_consensus.cc:3060] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.787690 17218 ts_tablet_manager.cc:1434] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:03:44.799520 17218 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.802276 16850 catalog_manager.cc:5697] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.807597 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "117045952bb14176abd739933fa51259" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:44.776429 17318 raft_consensus.cc:515] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.808300 16950 raft_consensus.cc:3060] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.810716 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:44.811435 17098 raft_consensus.cc:2468] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.816370 17141 leader_election.cc:304] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.818181 17263 raft_consensus.cc:2804] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:44.818557 17263 raft_consensus.cc:493] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.818881 17263 raft_consensus.cc:3060] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.817420 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:44.821185 17025 raft_consensus.cc:2468] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:44.827919 16950 raft_consensus.cc:2468] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:44.828457 17100 raft_consensus.cc:2468] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:44.832505 16992 leader_election.cc:304] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:44.833410 17467 raft_consensus.cc:2804] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.833932 17467 raft_consensus.cc:697] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:44.834707 17263 raft_consensus.cc:515] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.834935 17467 consensus_queue.cc:237] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:44.839107 17238 raft_consensus.cc:359] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.857975 17238 raft_consensus.cc:385] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.858325 17238 raft_consensus.cc:740] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.856949 17460 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.846792 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ce161acdcd435a8f869ae50a6ea162" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:44.859081 16950 raft_consensus.cc:3060] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.859789 17218 tablet_bootstrap.cc:654] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.848537 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ce161acdcd435a8f869ae50a6ea162" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:44.862498 17141 leader_election.cc:343] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.862748 17438 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.862419 17238 consensus_queue.cc:260] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.849931 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.865525 16845 catalog_manager.cc:5697] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.866391 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:44.866604 17026 raft_consensus.cc:3060] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:44.867865 16918 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.848939 17219 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:44.868943 17219 ts_tablet_manager.cc:1403] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.211s	user 0.019s	sys 0.009s
I20260709 13:03:44.869118 17238 ts_tablet_manager.cc:1434] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.001s
I20260709 13:03:44.870188 17238 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:44.871543 17438 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:44.851708 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e797287dc34d42e3838f8f5a3ba68985" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.874063 17099 raft_consensus.cc:3060] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.850636 17263 leader_election.cc:290] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.865648 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.874524 17219 raft_consensus.cc:359] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.875370 17219 raft_consensus.cc:385] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.875429 17100 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.875758 17219 raft_consensus.cc:740] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.876616 17219 consensus_queue.cc:260] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.850986 17220 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:44.847044 17318 leader_election.cc:290] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:44.878106 17220 ts_tablet_manager.cc:1403] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.212s	user 0.006s	sys 0.018s
I20260709 13:03:44.882543 17099 raft_consensus.cc:2468] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.883958 17141 leader_election.cc:304] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:44.884131 17218 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.884706 17218 ts_tablet_manager.cc:1403] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.000s
I20260709 13:03:44.884873 17318 raft_consensus.cc:2804] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.885418 17318 raft_consensus.cc:697] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:44.886233 17318 consensus_queue.cc:237] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.886802 17100 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:44.887552 17218 raft_consensus.cc:359] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.888448 17218 raft_consensus.cc:385] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.888708 17218 raft_consensus.cc:740] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.889401 16918 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:44.890298 17218 consensus_queue.cc:260] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:44.892478 17218 ts_tablet_manager.cc:1434] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:44.893392 17218 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.881071 17220 raft_consensus.cc:359] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.893877 17220 raft_consensus.cc:385] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.894193 17220 raft_consensus.cc:740] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.899072 17026 raft_consensus.cc:2468] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.908742 17460 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:44.900563 17319 consensus_queue.cc:1048] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.899410 17302 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.910750 17302 raft_consensus.cc:697] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:44.911630 17302 consensus_queue.cc:237] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:44.901197 17141 consensus_peers.cc:597] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:44.913472 17218 tablet_bootstrap.cc:654] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.913194 17343 consensus_queue.cc:1048] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.918741 17433 consensus_queue.cc:1048] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:44.928704 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:44.929023 17219 ts_tablet_manager.cc:1434] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.001s
I20260709 13:03:44.932353 17219 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:44.902419 17220 consensus_queue.cc:260] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.935200 17220 ts_tablet_manager.cc:1434] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:03:44.938004 17220 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:44.943259 17218 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:44.943828 17218 ts_tablet_manager.cc:1403] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.051s	user 0.002s	sys 0.010s
I20260709 13:03:44.946614 17218 raft_consensus.cc:359] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.947326 17218 raft_consensus.cc:385] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:44.947593 17218 raft_consensus.cc:740] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:44.948226 17218 consensus_queue.cc:260] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:44.949669 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:44.950323 17100 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:44.953560 17219 tablet_bootstrap.cc:654] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:44.936558 17238 tablet_bootstrap.cc:654] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:44.936772 17141 leader_election.cc:343] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:44.897681 16950 raft_consensus.cc:2468] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.956660 17140 leader_election.cc:304] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:44.958416 17343 raft_consensus.cc:2804] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:44.958925 17343 raft_consensus.cc:697] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:44.959136 17218 ts_tablet_manager.cc:1434] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:03:44.960227 17218 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:44.959779 17343 consensus_queue.cc:237] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:44.978945 17220 tablet_bootstrap.cc:654] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:44.966480 16992 consensus_peers.cc:597] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:44.976763 16850 catalog_manager.cc:5697] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:44.964200 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:44.987639 17100 raft_consensus.cc:2468] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:44.999650 17220 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.000531 17220 ts_tablet_manager.cc:1403] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.005s
I20260709 13:03:44.994185 17532 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.023144 17220 raft_consensus.cc:359] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.023916 17220 raft_consensus.cc:385] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.024315 17220 raft_consensus.cc:740] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.025334 17220 consensus_queue.cc:260] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.030794 17141 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.029793 17220 ts_tablet_manager.cc:1434] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.003s
I20260709 13:03:45.042085 17263 raft_consensus.cc:2804] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:45.042616 17263 raft_consensus.cc:697] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:45.041714 17220 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.043356 17263 consensus_queue.cc:237] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.051573 17220 tablet_bootstrap.cc:654] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.060073 17218 tablet_bootstrap.cc:654] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.061488 17219 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:45.062311 17219 ts_tablet_manager.cc:1403] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.130s	user 0.030s	sys 0.022s
I20260709 13:03:45.032521 17532 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.070187 17467 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.081701 17238 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:45.083554 17532 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.085106 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.093484 17238 ts_tablet_manager.cc:1403] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.223s	user 0.018s	sys 0.025s
I20260709 13:03:45.094640 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.099462 17460 consensus_queue.cc:1048] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.109619 17467 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.111935 17319 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.117547 17319 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.119690 17319 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.119227 17238 raft_consensus.cc:359] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.120242 17238 raft_consensus.cc:385] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.120580 17238 raft_consensus.cc:740] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.120615 16950 raft_consensus.cc:2468] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:45.121767 17238 consensus_queue.cc:260] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.124883 17175 raft_consensus.cc:3060] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.128839 17219 raft_consensus.cc:359] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.129522 17467 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:45.131971 17140 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:45.129637 17219 raft_consensus.cc:385] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.133226 17219 raft_consensus.cc:740] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.160035 17238 ts_tablet_manager.cc:1434] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.002s
I20260709 13:03:45.161547 17238 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:45.165495 17219 consensus_queue.cc:260] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.155743 17543 raft_consensus.cc:2804] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:45.171851 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 68263us
W20260709 13:03:45.188454 17141 leader_election.cc:343] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.158501 17466 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.195964 17466 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.198092 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:45.198839 17101 raft_consensus.cc:2468] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:45.200626 17466 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:45.180711 17543 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.207906 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 112666us
I20260709 13:03:45.210729 17219 ts_tablet_manager.cc:1434] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.148s	user 0.008s	sys 0.000s
I20260709 13:03:45.212126 17219 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:45.213444 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:45.214118 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:45.219110 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.220148 17220 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.220669 17220 ts_tablet_manager.cc:1403] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.186s	user 0.020s	sys 0.012s
I20260709 13:03:45.204988 17543 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.222601 17169 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.230032 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:45.234267 17067 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.241508 16993 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:45.241910 17262 consensus_queue.cc:1048] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.248832 17262 raft_consensus.cc:2804] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:45.249217 17262 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.249586 17262 raft_consensus.cc:3060] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.250674 17505 raft_consensus.cc:493] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.251165 17505 raft_consensus.cc:515] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.247259 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.258589 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.255085 16850 catalog_manager.cc:5697] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:45.262154 17067 leader_election.cc:343] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.261533 17218 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:45.265653 17218 ts_tablet_manager.cc:1403] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.306s	user 0.020s	sys 0.019s
I20260709 13:03:45.265326 17220 raft_consensus.cc:359] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.266114 17220 raft_consensus.cc:385] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.266396 17220 raft_consensus.cc:740] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
W20260709 13:03:45.253760 16993 leader_election.cc:343] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.253266 17505 leader_election.cc:290] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
W20260709 13:03:45.274712 17066 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.275236 17066 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:45.275977 17341 raft_consensus.cc:2749] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.267757 17220 consensus_queue.cc:260] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:45.290977 16918 leader_election.cc:343] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.268378 17218 raft_consensus.cc:359] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.294315 17218 raft_consensus.cc:385] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.294653 17218 raft_consensus.cc:740] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.304781 17218 consensus_queue.cc:260] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.311609 17219 tablet_bootstrap.cc:654] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.318145 17220 ts_tablet_manager.cc:1434] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 13:03:45.319665 17220 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.323676 17218 ts_tablet_manager.cc:1434] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 13:03:45.325052 17218 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:45.327303 17173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:45.313967 17467 raft_consensus.cc:493] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.337226 17547 consensus_queue.cc:1048] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:03:45.309719 17543 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.345965 17467 raft_consensus.cc:515] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:45.358912 17067 leader_election.cc:343] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.360101 17067 leader_election.cc:304] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.349390 17262 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.371572 17262 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:45.364207 17341 raft_consensus.cc:2749] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.379381 17218 tablet_bootstrap.cc:654] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.384296 17543 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.382489 17170 raft_consensus.cc:2393] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:45.388418 17219 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:45.389662 17219 ts_tablet_manager.cc:1403] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.178s	user 0.011s	sys 0.011s
I20260709 13:03:45.392833 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:45.393595 17101 raft_consensus.cc:3060] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.404688 16948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:45.404915 16947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.405467 16948 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.407763 17219 raft_consensus.cc:359] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.408497 17219 raft_consensus.cc:385] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.408746 17219 raft_consensus.cc:740] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
W20260709 13:03:45.411827 16992 leader_election.cc:343] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.413130 16948 raft_consensus.cc:2468] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:45.416328 17140 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
W20260709 13:03:45.413456 17554 log.cc:927] Time spent T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.003s
I20260709 13:03:45.414672 17545 consensus_queue.cc:1048] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:03:45.419555 17343 raft_consensus.cc:2804] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:45.420113 17343 raft_consensus.cc:697] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:45.421736 17460 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.422258 17460 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.420907 17101 raft_consensus.cc:2468] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:45.420886 17343 consensus_queue.cc:237] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.434386 17467 leader_election.cc:290] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:45.434800 17460 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.439440 17548 consensus_queue.cc:1048] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:45.427726 17171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:45.441399 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:45.441633 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.442665 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:45.443369 17097 raft_consensus.cc:2468] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:45.429023 17238 tablet_bootstrap.cc:654] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.447865 16948 raft_consensus.cc:3060] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.424702 17219 consensus_queue.cc:260] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:45.451004 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:45.467674 17263 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.468081 17220 tablet_bootstrap.cc:654] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.468420 17263 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.469832 16918 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.481715 17545 raft_consensus.cc:2749] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.484836 17263 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:03:45.488300 16993 leader_election.cc:343] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.490094 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.491091 16952 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:45.496650 17140 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:45.498322 17219 ts_tablet_manager.cc:1434] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.108s	user 0.005s	sys 0.001s
I20260709 13:03:45.499262 17219 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:45.502831 17548 consensus_queue.cc:1048] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:45.506152 17561 log.cc:927] Time spent T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:45.508301 17263 raft_consensus.cc:2804] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:45.508760 17263 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.509081 17263 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.517597 16993 leader_election.cc:304] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:45.518662 17511 raft_consensus.cc:2804] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:45.519058 17511 raft_consensus.cc:493] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.519501 17511 raft_consensus.cc:3060] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.519976 17543 consensus_queue.cc:1048] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.527489 17263 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:45.533176 16918 consensus_peers.cc:597] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c -> Peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Couldn't send request to peer 8267993b082f449891e464bc4dee65f7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.539453 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:45.540113 16952 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.549762 16952 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:45.552239 17557 consensus_queue.cc:1048] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:45.556341 17140 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:45.557432 17543 raft_consensus.cc:2804] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:45.558070 17543 raft_consensus.cc:697] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:45.559489 17543 consensus_queue.cc:237] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.571897 16847 catalog_manager.cc:5697] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.575407 16847 catalog_manager.cc:5697] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:45.578436 17263 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.519069 16993 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:45.597270 17220 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.597270 17238 tablet_bootstrap.cc:492] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:45.598730 17238 ts_tablet_manager.cc:1403] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.437s	user 0.020s	sys 0.029s
I20260709 13:03:45.597817 17220 ts_tablet_manager.cc:1403] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.278s	user 0.009s	sys 0.006s
I20260709 13:03:45.597807 17511 raft_consensus.cc:515] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:45.601986 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 83012.9us
I20260709 13:03:45.602108 17341 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.604163 17218 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:45.608740 17218 ts_tablet_manager.cc:1403] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.284s	user 0.028s	sys 0.018s
I20260709 13:03:45.610050 17220 raft_consensus.cc:359] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.610710 17220 raft_consensus.cc:385] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.598280 17467 raft_consensus.cc:2804] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:45.610934 17220 raft_consensus.cc:740] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.599107 17318 consensus_queue.cc:1048] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:03:45.616874 17220 consensus_queue.cc:260] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.619170 17220 ts_tablet_manager.cc:1434] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.001s
I20260709 13:03:45.620134 17220 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.612715 17511 leader_election.cc:290] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:45.611390 17341 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.622752 17218 raft_consensus.cc:359] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.623481 17218 raft_consensus.cc:385] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.623775 17218 raft_consensus.cc:740] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.623988 17341 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.625718 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.626299 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.626550 17218 consensus_queue.cc:260] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:45.627238 17066 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.628742 17218 ts_tablet_manager.cc:1434] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:03:45.615434 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:45.621778 17343 consensus_queue.cc:1048] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.631920 17467 raft_consensus.cc:697] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
W20260709 13:03:45.632229 17067 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.632702 17067 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.632896 17467 consensus_queue.cc:237] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:45.634028 16992 leader_election.cc:343] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.634792 17341 raft_consensus.cc:2749] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:45.640026 17141 leader_election.cc:343] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.644570 17218 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:45.639998 17238 raft_consensus.cc:359] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.645370 17238 raft_consensus.cc:385] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.645637 17238 raft_consensus.cc:740] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.646212 17238 consensus_queue.cc:260] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.647922 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.656488 17220 tablet_bootstrap.cc:654] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.662549 17219 tablet_bootstrap.cc:654] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.667587 17238 ts_tablet_manager.cc:1434] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:03:45.668951 17238 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:45.699360 16850 catalog_manager.cc:5697] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.710502 16850 catalog_manager.cc:5697] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.715473 17218 tablet_bootstrap.cc:654] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.725715 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.726748 17024 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:45.729954 17238 tablet_bootstrap.cc:654] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.758349 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:45.759125 17022 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.761926 17021 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.699532 17547 raft_consensus.cc:493] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:03:45.762552 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12c1a9abfa94488a9aefb37cb9ecdf0" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:45.763214 17101 raft_consensus.cc:3060] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.763039 17547 raft_consensus.cc:515] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.772897 17141 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.775055 17220 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.775593 17220 ts_tablet_manager.cc:1403] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.156s	user 0.014s	sys 0.011s
I20260709 13:03:45.775105 17543 raft_consensus.cc:2749] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:45.783473 17220 raft_consensus.cc:359] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.784242 17220 raft_consensus.cc:385] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.784535 17220 raft_consensus.cc:740] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
W20260709 13:03:45.793484 17141 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.793642 17318 consensus_queue.cc:1048] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.796175 17547 leader_election.cc:290] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:45.785210 17220 consensus_queue.cc:260] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.797263 17022 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:45.802315 17220 ts_tablet_manager.cc:1434] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:45.806242 17220 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.806640 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.810184 17263 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.810854 17263 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.817462 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0395376e4b3d44a0a28a7bb199a21307" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:45.818645 17263 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.822844 16918 leader_election.cc:304] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.825798 17545 raft_consensus.cc:2749] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:45.828296 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.829607 17101 raft_consensus.cc:2468] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:45.834921 17263 consensus_queue.cc:1048] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:45.839792 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:45.845089 17238 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:45.845667 17238 ts_tablet_manager.cc:1403] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.177s	user 0.019s	sys 0.005s
W20260709 13:03:45.846422 17140 leader_election.cc:343] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.848091 16993 leader_election.cc:304] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:45.848537 17238 raft_consensus.cc:359] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.853035 17220 tablet_bootstrap.cc:654] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.867652 17218 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:45.849232 17238 raft_consensus.cc:385] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.868212 17218 ts_tablet_manager.cc:1403] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.224s	user 0.012s	sys 0.000s
I20260709 13:03:45.859871 17219 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:45.864908 17511 raft_consensus.cc:2804] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:45.871034 17511 raft_consensus.cc:697] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:45.871510 17218 raft_consensus.cc:359] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.871953 17511 consensus_queue.cc:237] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.872332 17218 raft_consensus.cc:385] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.872735 17218 raft_consensus.cc:740] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.873533 17218 consensus_queue.cc:260] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:45.865573 17141 leader_election.cc:343] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.875025 17141 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.875891 17263 raft_consensus.cc:2749] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.876003 17218 ts_tablet_manager.cc:1434] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:45.868157 17238 raft_consensus.cc:740] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.877175 17218 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:45.877954 17238 consensus_queue.cc:260] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.881636 17219 ts_tablet_manager.cc:1403] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.383s	user 0.009s	sys 0.015s
I20260709 13:03:45.883080 17238 ts_tablet_manager.cc:1434] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:03:45.862793 17460 consensus_queue.cc:1048] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:45.884577 17218 tablet_bootstrap.cc:654] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.886683 17219 raft_consensus.cc:359] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.887470 17219 raft_consensus.cc:385] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.887742 17219 raft_consensus.cc:740] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.888454 17219 consensus_queue.cc:260] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.890395 17238 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:45.891419 17220 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.891978 17220 ts_tablet_manager.cc:1403] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.001s
I20260709 13:03:45.894835 17220 raft_consensus.cc:359] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.890385 16845 catalog_manager.cc:5697] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.895584 17220 raft_consensus.cc:385] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.895915 17220 raft_consensus.cc:740] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.899682 17543 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.899976 17219 ts_tablet_manager.cc:1434] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:03:45.900250 17543 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.901014 17219 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:45.902400 17543 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:45.896700 17220 consensus_queue.cc:260] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.905226 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:45.907020 17220 ts_tablet_manager.cc:1434] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:45.908027 17220 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.909592 17238 tablet_bootstrap.cc:654] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.912352 17547 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.912873 17547 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.915762 17219 tablet_bootstrap.cc:654] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:45.917392 17141 consensus_peers.cc:597] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.921172 17220 tablet_bootstrap.cc:654] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.919731 17547 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.921587 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:45.923657 16918 leader_election.cc:343] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.937490 17220 tablet_bootstrap.cc:492] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:45.938099 17220 ts_tablet_manager.cc:1403] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.004s
I20260709 13:03:45.940233 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:45.941041 17238 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:45.941224 17170 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:45.941596 17238 ts_tablet_manager.cc:1403] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.002s
I20260709 13:03:45.943495 17219 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:45.943984 17219 ts_tablet_manager.cc:1403] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.007s
I20260709 13:03:45.944053 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.946869 16918 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:45.947203 17219 raft_consensus.cc:359] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.947902 17219 raft_consensus.cc:385] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.948477 17557 raft_consensus.cc:2804] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:45.941310 17220 raft_consensus.cc:359] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.949035 17557 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.949198 17220 raft_consensus.cc:385] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.949470 17557 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:45.950949 17238 raft_consensus.cc:359] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.949558 17220 raft_consensus.cc:740] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.952358 17220 consensus_queue.cc:260] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.948540 17219 raft_consensus.cc:740] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.953723 17219 consensus_queue.cc:260] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.956302 17219 ts_tablet_manager.cc:1434] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:03:45.957924 17466 raft_consensus.cc:493] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.958475 17466 raft_consensus.cc:515] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:45.958959 17218 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:45.959050 17220 ts_tablet_manager.cc:1434] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:03:45.959549 17218 ts_tablet_manager.cc:1403] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.083s	user 0.027s	sys 0.024s
I20260709 13:03:45.959718 17219 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:45.951679 17238 raft_consensus.cc:385] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.961453 17238 raft_consensus.cc:740] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:45.961252 17557 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.962515 17218 raft_consensus.cc:359] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:45.962257 17238 consensus_queue.cc:260] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:45.963153 17466 leader_election.cc:290] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:45.963364 17218 raft_consensus.cc:385] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:45.964334 17220 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:45.964664 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:45.963681 17218 raft_consensus.cc:740] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
W20260709 13:03:45.966143 17067 leader_election.cc:343] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.966441 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:45.966354 17218 consensus_queue.cc:260] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:45.968217 17066 leader_election.cc:343] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.968731 17218 ts_tablet_manager.cc:1434] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:45.968685 17066 leader_election.cc:304] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:45.969558 17557 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:45.970029 17466 raft_consensus.cc:2749] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:45.970184 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
W20260709 13:03:45.971475 16918 leader_election.cc:343] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:45.972025 17218 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: Bootstrap starting.
I20260709 13:03:45.973613 17219 tablet_bootstrap.cc:654] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.976161 17238 ts_tablet_manager.cc:1434] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.002s
I20260709 13:03:45.980679 17220 tablet_bootstrap.cc:654] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.984091 17218 tablet_bootstrap.cc:654] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:45.986729 17238 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:45.988300 17263 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.997916 17263 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:45.998479 17218 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: No bootstrap required, opened a new log
I20260709 13:03:45.999032 17218 ts_tablet_manager.cc:1403] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.009s
I20260709 13:03:45.999961 17263 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:46.000859 17140 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:46.001046 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.001816 17219 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.000131 17220 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.001984 17101 raft_consensus.cc:2393] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 1: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:46.002533 17219 ts_tablet_manager.cc:1403] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.000s
I20260709 13:03:46.001901 17218 raft_consensus.cc:359] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.002944 17218 raft_consensus.cc:385] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.003218 17218 raft_consensus.cc:740] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.003985 17218 consensus_queue.cc:260] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.004974 17219 raft_consensus.cc:359] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.005630 17219 raft_consensus.cc:385] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.005856 17219 raft_consensus.cc:740] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.006137 17218 ts_tablet_manager.cc:1434] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:46.006543 17219 consensus_queue.cc:260] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.008332 17219 ts_tablet_manager.cc:1434] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:03:46.008882 17220 ts_tablet_manager.cc:1403] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.008s
I20260709 13:03:46.010845 17263 raft_consensus.cc:2749] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.013134 17026 raft_consensus.cc:3060] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.014701 17220 raft_consensus.cc:359] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.015443 17220 raft_consensus.cc:385] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.015723 17220 raft_consensus.cc:740] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.016428 17238 tablet_bootstrap.cc:654] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.016419 17220 consensus_queue.cc:260] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.017179 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:46.018673 17220 ts_tablet_manager.cc:1434] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:03:46.019666 17220 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.022823 17219 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.023144 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:46.023813 17175 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.024468 17141 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.025972 17263 raft_consensus.cc:3060] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.028234 17238 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.028734 17238 ts_tablet_manager.cc:1403] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.005s
I20260709 13:03:46.031615 17219 tablet_bootstrap.cc:654] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.034940 17170 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.035840 16847 catalog_manager.cc:5697] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.038134 17220 tablet_bootstrap.cc:654] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.033010 17263 raft_consensus.cc:2749] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.042845 17175 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:46.048357 17220 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.049017 17220 ts_tablet_manager.cc:1403] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.002s
I20260709 13:03:46.055584 16918 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.057133 17547 raft_consensus.cc:2804] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.057682 17547 raft_consensus.cc:697] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:46.058527 17547 consensus_queue.cc:237] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.078289 17341 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.079003 17341 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.086586 17219 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.087158 17219 ts_tablet_manager.cc:1403] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.021s
I20260709 13:03:46.090590 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.091233 17556 consensus_queue.cc:1048] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.092020 17341 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.093669 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:46.095145 17066 leader_election.cc:343] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:46.098197 17067 leader_election.cc:343] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.098670 17067 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:46.099468 17341 raft_consensus.cc:2749] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.100644 17511 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.101344 17511 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.103818 17511 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.106494 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.107877 17341 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.106420 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.108412 17341 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.108692 17220 raft_consensus.cc:359] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.109490 17220 raft_consensus.cc:385] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.109833 17220 raft_consensus.cc:740] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.110689 17220 consensus_queue.cc:260] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.111285 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:46.111864 17341 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
W20260709 13:03:46.112736 17067 leader_election.cc:343] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.113123 17238 raft_consensus.cc:359] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.113814 17238 raft_consensus.cc:385] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.114087 17238 raft_consensus.cc:740] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.114023 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:03:46.115520 17067 leader_election.cc:343] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.118177 16993 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:46.119002 17511 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.115937 17067 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.121500 17220 ts_tablet_manager.cc:1434] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 13:03:46.124728 17341 raft_consensus.cc:2749] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.130486 17466 raft_consensus.cc:493] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.131034 17466 raft_consensus.cc:515] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.133093 17466 leader_election.cc:290] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:46.134956 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.125778 17220 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.136879 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:46.138680 17066 leader_election.cc:343] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.121002 16846 catalog_manager.cc:5697] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.130419 17219 raft_consensus.cc:359] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.141952 17219 raft_consensus.cc:385] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.142233 17219 raft_consensus.cc:740] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.142874 17219 consensus_queue.cc:260] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.145068 17219 ts_tablet_manager.cc:1434] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.000s
I20260709 13:03:46.146252 17219 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
W20260709 13:03:46.149186 17067 leader_election.cc:343] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.149482 17263 consensus_queue.cc:1048] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.149809 17067 leader_election.cc:304] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.150595 17466 raft_consensus.cc:2749] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.151021 17238 consensus_queue.cc:260] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.155534 17547 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.158555 17547 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.159467 17220 tablet_bootstrap.cc:654] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.161656 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.161955 17547 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.162341 17101 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:46.163565 16918 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:46.164098 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.164444 17547 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.164896 17547 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.165239 17547 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:46.166373 16918 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.178288 17219 tablet_bootstrap.cc:654] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.185300 17238 ts_tablet_manager.cc:1434] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.156s	user 0.009s	sys 0.000s
I20260709 13:03:46.186403 17238 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.190851 17467 consensus_queue.cc:1048] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.193092 17220 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.193728 17220 ts_tablet_manager.cc:1403] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.005s
I20260709 13:03:46.196249 17220 raft_consensus.cc:359] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.197146 17220 raft_consensus.cc:385] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.197474 17220 raft_consensus.cc:740] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.198195 17220 consensus_queue.cc:260] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.207067 17220 ts_tablet_manager.cc:1434] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:03:46.208045 17220 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.211655 17238 tablet_bootstrap.cc:654] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.220904 17547 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.226418 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:46.226810 17219 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.227084 17101 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.227734 17219 ts_tablet_manager.cc:1403] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.003s
I20260709 13:03:46.230145 17220 tablet_bootstrap.cc:654] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.230508 17219 raft_consensus.cc:359] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.231220 17219 raft_consensus.cc:385] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.231523 17219 raft_consensus.cc:740] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.232569 17219 consensus_queue.cc:260] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.232389 17547 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.237136 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:46.237768 17175 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.237965 17219 ts_tablet_manager.cc:1434] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:03:46.239104 17219 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.244951 17219 tablet_bootstrap.cc:654] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.249661 17175 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:46.253929 17547 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.255695 17547 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.257794 17547 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.255025 17101 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:46.260459 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.261121 17098 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:46.262035 16918 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:46.265005 16918 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:46.264983 17467 consensus_queue.cc:1048] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:46.266078 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.267771 17547 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.268283 17547 raft_consensus.cc:697] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:46.269153 17220 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.269724 17220 ts_tablet_manager.cc:1403] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.004s
I20260709 13:03:46.274837 17545 raft_consensus.cc:2804] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.269109 17547 consensus_queue.cc:237] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.275285 17545 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.276463 17220 raft_consensus.cc:359] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.277038 17545 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.277168 17220 raft_consensus.cc:385] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.277496 17220 raft_consensus.cc:740] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
W20260709 13:03:46.278491 16918 leader_election.cc:343] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.279217 17220 consensus_queue.cc:260] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.283962 17220 ts_tablet_manager.cc:1434] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:46.285727 17220 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.296145 17318 raft_consensus.cc:493] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.307512 17220 tablet_bootstrap.cc:654] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.307082 17318 raft_consensus.cc:515] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.309568 17318 leader_election.cc:290] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:46.310931 17219 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.311497 17219 ts_tablet_manager.cc:1403] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.009s
I20260709 13:03:46.327385 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
W20260709 13:03:46.333204 17141 leader_election.cc:343] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.348300 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.349000 16952 raft_consensus.cc:2468] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:46.350417 17140 leader_election.cc:304] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.337980 17511 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.353652 17263 raft_consensus.cc:2804] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.354043 17263 raft_consensus.cc:493] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.354355 17263 raft_consensus.cc:3060] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.354882 17262 consensus_queue.cc:1048] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:46.355786 17511 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.361258 17545 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.365366 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.365661 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.366190 17175 raft_consensus.cc:2393] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 1: Already voted for candidate 64734f14219348f59d1cf8700416565c in this term.
I20260709 13:03:46.367674 17021 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.362614 17219 raft_consensus.cc:359] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.368542 16992 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:46.362709 17238 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.368584 17219 raft_consensus.cc:385] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.369333 17219 raft_consensus.cc:740] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.369495 17238 ts_tablet_manager.cc:1403] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.183s	user 0.026s	sys 0.012s
I20260709 13:03:46.364351 17263 raft_consensus.cc:515] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.364632 17511 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.372582 17558 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.372720 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:46.373457 16951 raft_consensus.cc:3060] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.376543 17545 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.380362 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:46.381027 17101 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.381400 17263 leader_election.cc:290] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:46.370165 17219 consensus_queue.cc:260] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.387197 16850 catalog_manager.cc:5697] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.389786 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:46.394760 17558 raft_consensus.cc:2749] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.394672 17238 raft_consensus.cc:359] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.395520 17238 raft_consensus.cc:385] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.395803 17238 raft_consensus.cc:740] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.396512 17238 consensus_queue.cc:260] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:46.397744 17141 leader_election.cc:343] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.398139 17219 ts_tablet_manager.cc:1434] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.001s
I20260709 13:03:46.398221 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:46.398813 16951 raft_consensus.cc:2468] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.398901 17175 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.405138 17219 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.408375 17140 leader_election.cc:304] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.410573 17543 raft_consensus.cc:2804] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.410718 17101 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:46.411132 17543 raft_consensus.cc:697] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:46.412041 17543 consensus_queue.cc:237] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.414392 16918 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:46.415293 17545 raft_consensus.cc:2804] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.415807 17545 raft_consensus.cc:697] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:46.416405 17238 ts_tablet_manager.cc:1434] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.004s
I20260709 13:03:46.419670 17219 tablet_bootstrap.cc:654] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.426774 17175 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:46.416661 17545 consensus_queue.cc:237] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.426820 17238 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.463657 17219 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.464237 17219 ts_tablet_manager.cc:1403] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.024s
I20260709 13:03:46.468324 17238 tablet_bootstrap.cc:654] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.470554 17220 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.471060 17220 ts_tablet_manager.cc:1403] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.186s	user 0.022s	sys 0.009s
I20260709 13:03:46.471362 16952 raft_consensus.cc:3060] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.473222 17219 raft_consensus.cc:359] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.473929 17219 raft_consensus.cc:385] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.474210 17219 raft_consensus.cc:740] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.474836 17219 consensus_queue.cc:260] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:03:46.477008 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 45150.9us
I20260709 13:03:46.478386 17220 raft_consensus.cc:359] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.479074 17220 raft_consensus.cc:385] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.479359 17220 raft_consensus.cc:740] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.485713 17219 ts_tablet_manager.cc:1434] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:03:46.486689 17219 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.488579 17558 consensus_queue.cc:1048] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.480023 17220 consensus_queue.cc:260] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.495550 17220 ts_tablet_manager.cc:1434] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:03:46.496490 17220 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.511121 17263 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.511690 17263 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.524715 17263 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:46.527823 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:46.541496 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.542256 16951 raft_consensus.cc:2468] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:46.545864 17238 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.546497 17238 ts_tablet_manager.cc:1403] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.007s
I20260709 13:03:46.556973 17220 tablet_bootstrap.cc:654] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:46.557564 16898 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:03:46.545749 17141 leader_election.cc:343] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.564488 17140 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.565313 17543 raft_consensus.cc:2804] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.565721 17543 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.566038 17543 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.567611 17219 tablet_bootstrap.cc:654] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.570021 16850 catalog_manager.cc:5697] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.576849 17220 tablet_bootstrap.cc:492] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.577385 17220 ts_tablet_manager.cc:1403] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.003s
I20260709 13:03:46.580543 17220 raft_consensus.cc:359] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.581200 17220 raft_consensus.cc:385] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.581475 17220 raft_consensus.cc:740] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.582180 17220 consensus_queue.cc:260] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.583774 16847 catalog_manager.cc:5697] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.584614 17220 ts_tablet_manager.cc:1434] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:46.574249 17543 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.549204 17238 raft_consensus.cc:359] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.585525 17238 raft_consensus.cc:385] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.585816 17238 raft_consensus.cc:740] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.586491 17238 consensus_queue.cc:260] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.576038 17318 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.587625 17318 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.588353 17543 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:46.588608 17238 ts_tablet_manager.cc:1434] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.000s
I20260709 13:03:46.589532 17318 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.592520 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.594120 16952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:46.594738 16952 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.589746 17220 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: Bootstrap starting.
I20260709 13:03:46.593175 17100 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:46.590858 17238 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.598896 17318 raft_consensus.cc:493] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.589838 17021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:46.599421 17318 raft_consensus.cc:515] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:46.600323 17141 leader_election.cc:343] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.594575 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.601301 16951 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:46.601533 17318 leader_election.cc:290] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.604820 16952 raft_consensus.cc:2468] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.605638 17238 tablet_bootstrap.cc:654] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.605835 17220 tablet_bootstrap.cc:654] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.591850 17219 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.608557 17219 ts_tablet_manager.cc:1403] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.000s
I20260709 13:03:46.611954 17219 raft_consensus.cc:359] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.612789 17219 raft_consensus.cc:385] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.613131 17219 raft_consensus.cc:740] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.613767 17219 consensus_queue.cc:260] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.615936 17141 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:46.616897 17543 raft_consensus.cc:2804] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.617305 17543 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.618748 17543 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.618978 17467 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.619483 17467 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.622134 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.622774 17100 raft_consensus.cc:2468] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 0.
I20260709 13:03:46.625114 17141 leader_election.cc:304] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:46.626240 17467 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.627127 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.627463 17460 raft_consensus.cc:2804] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.627852 17460 raft_consensus.cc:493] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.627938 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.628235 17460 raft_consensus.cc:3060] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.630085 17175 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:46.630646 17543 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.631117 16993 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:46.632705 17543 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.633512 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.634341 17100 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:46.638368 17545 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.616897 17219 ts_tablet_manager.cc:1434] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:46.640324 17219 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.643889 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:46.640393 17140 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:46.641170 17547 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.644495 17100 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.636844 17460 raft_consensus.cc:515] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.646541 17318 raft_consensus.cc:2804] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.647012 17318 raft_consensus.cc:697] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:46.647802 17318 consensus_queue.cc:237] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.650449 17460 leader_election.cc:290] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:46.651124 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
W20260709 13:03:46.651780 17140 leader_election.cc:343] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.651825 17101 raft_consensus.cc:3060] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.642586 17238 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.641835 17467 raft_consensus.cc:2804] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.653155 17467 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.653540 17467 raft_consensus.cc:3060] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.657835 17219 tablet_bootstrap.cc:654] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.662397 17220 tablet_bootstrap.cc:492] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: No bootstrap required, opened a new log
I20260709 13:03:46.662873 17220 ts_tablet_manager.cc:1403] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.001s
I20260709 13:03:46.666841 17219 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.667407 17219 ts_tablet_manager.cc:1403] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:03:46.670028 17238 ts_tablet_manager.cc:1403] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.001s
I20260709 13:03:46.670188 17219 raft_consensus.cc:359] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.670931 17219 raft_consensus.cc:385] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.671211 17219 raft_consensus.cc:740] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.682264 17101 raft_consensus.cc:2468] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.671948 17219 consensus_queue.cc:260] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.684175 17302 raft_consensus.cc:493] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.684456 17141 leader_election.cc:304] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:46.684644 17302 raft_consensus.cc:515] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.685942 17220 raft_consensus.cc:359] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.686968 17220 raft_consensus.cc:385] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.687243 17220 raft_consensus.cc:740] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.687842 17220 consensus_queue.cc:260] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.689911 17220 ts_tablet_manager.cc:1434] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:03:46.693220 17100 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.694705 17263 raft_consensus.cc:2804] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.694996 17545 consensus_queue.cc:1048] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.695441 17263 raft_consensus.cc:697] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:46.696442 17263 consensus_queue.cc:237] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:46.699023 17141 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:46.691468 17302 leader_election.cc:290] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.705868 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.707628 17101 raft_consensus.cc:2393] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64734f14219348f59d1cf8700416565c in current term 1: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:46.708002 17556 raft_consensus.cc:2804] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.708725 17556 raft_consensus.cc:697] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:46.709792 17556 consensus_queue.cc:237] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:46.713150 17140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 60246.3us
I20260709 13:03:46.680861 17238 raft_consensus.cc:359] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.729017 17238 raft_consensus.cc:385] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.729394 17238 raft_consensus.cc:740] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.733894 17219 ts_tablet_manager.cc:1434] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.001s
I20260709 13:03:46.730044 17238 consensus_queue.cc:260] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.741773 17219 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:46.751237 17238 ts_tablet_manager.cc:1434] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260709 13:03:46.754390 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:46.755069 16951 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.750768 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.763403 16918 leader_election.cc:304] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:46.764142 17558 raft_consensus.cc:493] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.764670 17558 raft_consensus.cc:515] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.764952 17545 raft_consensus.cc:3060] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.765772 16948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14794cbc74f4fcf8d3872e3463ad58d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:46.777848 17238 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.783828 16951 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.794593 17547 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.795080 17558 leader_election.cc:290] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:46.795215 17547 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.797120 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.798280 17101 raft_consensus.cc:2468] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:46.799371 16993 leader_election.cc:304] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
W20260709 13:03:46.793426 17627 log.cc:927] Time spent T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:03:46.792531 17545 raft_consensus.cc:2749] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:46.805207 17238 tablet_bootstrap.cc:654] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.806389 17547 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:46.776652 17467 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.803223 17558 raft_consensus.cc:2804] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.808629 17558 raft_consensus.cc:493] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.808995 17558 raft_consensus.cc:3060] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.812044 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:46.812044 16951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:46.809131 17467 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.813618 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:46.816571 17100 raft_consensus.cc:3060] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.816097 17169 raft_consensus.cc:3060] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.832839 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.833560 17101 raft_consensus.cc:2468] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 0.
I20260709 13:03:46.834026 17219 tablet_bootstrap.cc:654] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.835216 16918 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:46.844192 17100 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
W20260709 13:03:46.846096 16992 leader_election.cc:343] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:46.847030 16993 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:46.847280 17547 raft_consensus.cc:2804] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:46.849347 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:46.884290 17262 raft_consensus.cc:2804] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:46.868724 16850 catalog_manager.cc:5697] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.884836 17238 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.872654 17547 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.885398 17238 ts_tablet_manager.cc:1403] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.010s
I20260709 13:03:46.887370 16850 catalog_manager.cc:5697] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.891680 17238 raft_consensus.cc:359] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.892370 17238 raft_consensus.cc:385] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.892624 17238 raft_consensus.cc:740] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.884898 17262 raft_consensus.cc:697] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:46.893304 17238 consensus_queue.cc:260] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:46.893541 17262 consensus_queue.cc:237] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.894138 17625 consensus_queue.cc:1048] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:46.895454 17238 ts_tablet_manager.cc:1434] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.002s
I20260709 13:03:46.888381 17547 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.896373 16850 catalog_manager.cc:5697] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 1 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.925839 17238 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.932147 17238 tablet_bootstrap.cc:654] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:46.935663 17612 raft_consensus.cc:493] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.936316 17612 raft_consensus.cc:515] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.941804 17547 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.922500 16948 raft_consensus.cc:2468] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:46.932693 17558 raft_consensus.cc:515] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.950095 17219 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: No bootstrap required, opened a new log
I20260709 13:03:46.967073 17219 ts_tablet_manager.cc:1403] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.232s	user 0.037s	sys 0.039s
I20260709 13:03:46.954332 17238 tablet_bootstrap.cc:492] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:46.969204 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:46.969965 17238 ts_tablet_manager.cc:1403] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.044s	user 0.003s	sys 0.010s
I20260709 13:03:46.958982 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:46.958751 17171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:46.971179 17101 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:46.971505 17171 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 0.
I20260709 13:03:46.969905 17219 raft_consensus.cc:359] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.975093 17219 raft_consensus.cc:385] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.972923 17238 raft_consensus.cc:359] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:03:46.956041 17141 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:46.975773 17238 raft_consensus.cc:385] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:46.976202 17238 raft_consensus.cc:740] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.976934 17238 consensus_queue.cc:260] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:46.975394 17219 raft_consensus.cc:740] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:46.978826 17219 consensus_queue.cc:260] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:46.947630 17612 leader_election.cc:290] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:46.982666 17238 ts_tablet_manager.cc:1434] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:03:46.981138 17219 ts_tablet_manager.cc:1434] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:46.986893 17100 raft_consensus.cc:3060] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.965051 17558 leader_election.cc:290] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:46.964475 16948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17e18d23d343e6894f11dc02b2eb75" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:46.995517 17238 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:46.995926 16993 leader_election.cc:304] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:46.996668 16948 raft_consensus.cc:3060] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.965837 17169 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:47.002698 17467 raft_consensus.cc:2804] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:47.003120 17467 raft_consensus.cc:493] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.003455 17467 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:46.944635 17547 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:03:47.013108 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 133427us
I20260709 13:03:47.014508 17027 raft_consensus.cc:3060] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.017133 17467 raft_consensus.cc:515] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:47.020887 17467 leader_election.cc:290] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:03:47.024189 17171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:47.024372 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:47.024820 17171 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.024966 17101 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.027100 16948 raft_consensus.cc:2468] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:47.028745 17021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:47.031502 17096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:47.032123 17096 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.039062 17238 tablet_bootstrap.cc:654] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:47.039737 17171 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:47.041361 16992 leader_election.cc:304] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:47.042204 17612 raft_consensus.cc:2804] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:47.042754 17612 raft_consensus.cc:697] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:47.043666 17612 consensus_queue.cc:237] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:47.043421 16850 catalog_manager.cc:5697] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:47.055236 17101 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:47.061681 16993 leader_election.cc:304] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:47.062443 17467 raft_consensus.cc:2804] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:47.062938 17467 raft_consensus.cc:697] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:47.024919 17100 raft_consensus.cc:2468] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:47.073179 17096 raft_consensus.cc:2468] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 1.
I20260709 13:03:47.081144 17556 consensus_queue.cc:1048] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:47.095192 16918 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:47.096800 16918 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:47.071421 17467 consensus_queue.cc:237] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:03:47.122208 17262 raft_consensus.cc:493] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.122792 17262 raft_consensus.cc:515] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:47.129899 17238 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:47.130395 17238 ts_tablet_manager.cc:1403] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.025s
I20260709 13:03:47.111918 17638 consensus_queue.cc:1048] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:03:47.149716 17262 leader_election.cc:290] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:47.167941 16918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 65911.9us
I20260709 13:03:47.170683 16847 catalog_manager.cc:5697] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:47.178972 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:47.182518 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e1d0f0dc3240608259e975a8ea33ec" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:47.187912 16992 leader_election.cc:304] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:47.188725 17238 raft_consensus.cc:359] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:47.191016 17238 raft_consensus.cc:385] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:47.191284 17238 raft_consensus.cc:740] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:47.192070 17238 consensus_queue.cc:260] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:47.194168 17238 ts_tablet_manager.cc:1434] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 13:03:47.195039 17238 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:47.200344 17262 raft_consensus.cc:2749] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:47.213150 17238 tablet_bootstrap.cc:654] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:47.231504 16918 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:47.235970 17646 raft_consensus.cc:2804] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:47.236553 17646 raft_consensus.cc:697] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:03:47.237435 17646 consensus_queue.cc:237] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:47.247145 17638 consensus_queue.cc:1048] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:47.251351 17318 consensus_queue.cc:1048] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:47.256070 17556 consensus_queue.cc:1048] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:47.261528 16847 catalog_manager.cc:5697] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:47.281615 17141 consensus_peers.cc:597] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:47.282214 17238 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:47.282812 17238 ts_tablet_manager.cc:1403] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.088s	user 0.021s	sys 0.004s
I20260709 13:03:47.283934 17628 consensus_queue.cc:1048] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:47.286245 17238 raft_consensus.cc:359] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:47.287122 17238 raft_consensus.cc:385] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:47.287439 17238 raft_consensus.cc:740] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:47.288234 17238 consensus_queue.cc:260] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:47.290598 17025 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.301689 17238 ts_tablet_manager.cc:1434] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:03:47.302768 17238 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Bootstrap starting.
I20260709 13:03:47.308445 17238 tablet_bootstrap.cc:654] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:47.320376 17238 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: No bootstrap required, opened a new log
I20260709 13:03:47.320880 17238 ts_tablet_manager.cc:1403] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:03:47.333936 17238 raft_consensus.cc:359] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:47.334672 17238 raft_consensus.cc:385] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:47.334934 17238 raft_consensus.cc:740] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Initialized, Role: FOLLOWER
I20260709 13:03:47.335675 17238 consensus_queue.cc:260] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:47.320056 16847 catalog_manager.cc:5697] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 0 to 1, leader changed from <none> to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 1 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:47.339140 17238 ts_tablet_manager.cc:1434] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
W20260709 13:03:47.345425 17140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 93071.1us
I20260709 13:03:47.346839 17647 consensus_queue.cc:1048] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:47.459470 17662 log.cc:927] Time spent T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:03:47.476644 17141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 117139us
I20260709 13:03:47.515861 17467 consensus_queue.cc:1048] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:47.518750 17635 consensus_queue.cc:1048] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:47.533725 17656 consensus_queue.cc:1048] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:47.623353 16992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 98937.9us
W20260709 13:03:47.700384 16992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 74284.5us
I20260709 13:03:47.711398 17635 consensus_queue.cc:1048] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:47.732867 17639 consensus_queue.cc:1048] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:03:47.757097 17668 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.758230 17668 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:47.765131 17668 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:47.787490 17025 raft_consensus.cc:3060] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.831164 17027 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.880366 16947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:47.892777 17672 consensus_queue.cc:1048] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:03:47.901192 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:47.936569 17021 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:47.969916 16993 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:47.984884 17672 consensus_queue.cc:1048] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:48.009397 17558 consensus_queue.cc:1048] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:48.020992 17635 raft_consensus.cc:2749] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:48.047307 17669 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8267993b082f449891e464bc4dee65f7)
I20260709 13:03:48.047941 17669 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:48.068349 17669 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:48.070123 16950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:48.082317 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:48.097662 17545 consensus_queue.cc:1048] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:48.116290 16993 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:48.119541 17669 raft_consensus.cc:2749] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:49.258244  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
W20260709 13:03:49.262387 17066 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 13:03:49.299199 17066 consensus_peers.cc:597] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.303706 16992 consensus_peers.cc:597] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.305974 16992 consensus_peers.cc:597] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.311184 17140 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.315994 17140 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.341504  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:49.342623  7638 tablet_replica.cc:333] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.343901  7638 raft_consensus.cc:2243] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.344887  7638 raft_consensus.cc:2272] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.347433  7638 tablet_replica.cc:333] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.348483  7638 raft_consensus.cc:2243] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.349161  7638 raft_consensus.cc:2272] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.352259  7638 tablet_replica.cc:333] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.352952  7638 raft_consensus.cc:2243] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.353904  7638 raft_consensus.cc:2272] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.355997  7638 tablet_replica.cc:333] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.356601  7638 raft_consensus.cc:2243] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.357448  7638 raft_consensus.cc:2272] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.359552  7638 tablet_replica.cc:333] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.360175  7638 raft_consensus.cc:2243] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.360595  7638 raft_consensus.cc:2272] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.362804  7638 tablet_replica.cc:333] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.363473  7638 raft_consensus.cc:2243] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.363901  7638 raft_consensus.cc:2272] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.366638  7638 tablet_replica.cc:333] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.367293  7638 raft_consensus.cc:2243] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.367738  7638 raft_consensus.cc:2272] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.369633  7638 tablet_replica.cc:333] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.370214  7638 raft_consensus.cc:2243] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.370988  7638 raft_consensus.cc:2272] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.373425  7638 tablet_replica.cc:333] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.374234  7638 raft_consensus.cc:2243] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.374720  7638 raft_consensus.cc:2272] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.376987  7638 tablet_replica.cc:333] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.377662  7638 raft_consensus.cc:2243] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.378527  7638 raft_consensus.cc:2272] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.380661  7638 tablet_replica.cc:333] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.381213  7638 raft_consensus.cc:2243] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.381775  7638 raft_consensus.cc:2272] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.384048  7638 tablet_replica.cc:333] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.384814  7638 raft_consensus.cc:2243] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:03:49.385650 17140 consensus_peers.cc:597] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.385735  7638 raft_consensus.cc:2272] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.388217  7638 tablet_replica.cc:333] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.389063  7638 raft_consensus.cc:2243] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.390046  7638 raft_consensus.cc:2272] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.393585  7638 tablet_replica.cc:333] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.394641  7638 raft_consensus.cc:2243] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.395236  7638 raft_consensus.cc:2272] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.396956 17066 consensus_peers.cc:597] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.398167  7638 tablet_replica.cc:333] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.399003  7638 raft_consensus.cc:2243] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.400018  7638 raft_consensus.cc:2272] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.402990  7638 tablet_replica.cc:333] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.403712  7638 raft_consensus.cc:2243] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.404253  7638 raft_consensus.cc:2272] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.407033  7638 tablet_replica.cc:333] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.407742  7638 raft_consensus.cc:2243] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.408231  7638 raft_consensus.cc:2272] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.411041  7638 tablet_replica.cc:333] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.411885  7638 raft_consensus.cc:2243] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.412602  7638 raft_consensus.cc:2272] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.415277  7638 tablet_replica.cc:333] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.416101  7638 raft_consensus.cc:2243] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.416579  7638 raft_consensus.cc:2272] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.418942  7638 tablet_replica.cc:333] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.419700  7638 raft_consensus.cc:2243] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.421722  7638 raft_consensus.cc:2272] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.424266  7638 tablet_replica.cc:333] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.424855  7638 raft_consensus.cc:2243] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.425278  7638 raft_consensus.cc:2272] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.427158  7638 tablet_replica.cc:333] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.427685  7638 raft_consensus.cc:2243] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.428076  7638 raft_consensus.cc:2272] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.430290  7638 tablet_replica.cc:333] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.431022  7638 raft_consensus.cc:2243] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.432073  7638 raft_consensus.cc:2272] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.433421 16992 consensus_peers.cc:597] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.434468  7638 tablet_replica.cc:333] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.435120  7638 raft_consensus.cc:2243] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.435871  7638 raft_consensus.cc:2272] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.438519  7638 tablet_replica.cc:333] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.439306  7638 raft_consensus.cc:2243] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.439864  7638 raft_consensus.cc:2272] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.441659 17140 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.442322  7638 tablet_replica.cc:333] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.442919  7638 raft_consensus.cc:2243] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.443331  7638 raft_consensus.cc:2272] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.445801  7638 tablet_replica.cc:333] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.446380  7638 raft_consensus.cc:2243] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.446866  7638 raft_consensus.cc:2272] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.448928  7638 tablet_replica.cc:333] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.449558  7638 raft_consensus.cc:2243] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.450397  7638 raft_consensus.cc:2272] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.452502  7638 tablet_replica.cc:333] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.453059  7638 raft_consensus.cc:2243] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.453830  7638 raft_consensus.cc:2272] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.455658  7638 tablet_replica.cc:333] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.456270  7638 raft_consensus.cc:2243] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.456802  7638 raft_consensus.cc:2272] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.458943  7638 tablet_replica.cc:333] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.459453  7638 raft_consensus.cc:2243] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.460219  7638 raft_consensus.cc:2272] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.462072  7638 tablet_replica.cc:333] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.462620  7638 raft_consensus.cc:2243] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.463084  7638 raft_consensus.cc:2272] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.466638  7638 tablet_replica.cc:333] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.467475  7638 raft_consensus.cc:2243] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:49.467823 16992 consensus_peers.cc:597] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.468057  7638 raft_consensus.cc:2272] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.471014  7638 tablet_replica.cc:333] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.471544  7638 raft_consensus.cc:2243] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.472282  7638 raft_consensus.cc:2272] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.474479  7638 tablet_replica.cc:333] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.475061  7638 raft_consensus.cc:2243] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.476166  7638 raft_consensus.cc:2272] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.478292 16992 consensus_peers.cc:597] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.478760 17140 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.479035  7638 tablet_replica.cc:333] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.479684  7638 raft_consensus.cc:2243] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.480126  7638 raft_consensus.cc:2272] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.482218  7638 tablet_replica.cc:333] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.482731  7638 raft_consensus.cc:2243] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.483512  7638 raft_consensus.cc:2272] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.485605  7638 tablet_replica.cc:333] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.486291  7638 raft_consensus.cc:2243] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.486775  7638 raft_consensus.cc:2272] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.490046  7638 tablet_replica.cc:333] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.491044  7638 raft_consensus.cc:2243] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.491547  7638 raft_consensus.cc:2272] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.493355  7638 tablet_replica.cc:333] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.493857  7638 raft_consensus.cc:2243] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.494272  7638 raft_consensus.cc:2272] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.496564  7638 tablet_replica.cc:333] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.497370  7638 raft_consensus.cc:2243] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:49.497848 17140 consensus_peers.cc:597] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.497870  7638 raft_consensus.cc:2272] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.500097  7638 tablet_replica.cc:333] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.500665  7638 raft_consensus.cc:2243] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.501109  7638 raft_consensus.cc:2272] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.503640  7638 tablet_replica.cc:333] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.504258  7638 raft_consensus.cc:2243] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.504745  7638 raft_consensus.cc:2272] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:49.507109  7638 tablet_replica.cc:333] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: stopping tablet replica
W20260709 13:03:49.507429 16992 consensus_peers.cc:597] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.507580  7638 raft_consensus.cc:2243] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:49.508544  7638 raft_consensus.cc:2272] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.509845 16992 consensus_peers.cc:597] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.511023  7638 tablet_replica.cc:333] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: stopping tablet replica
I20260709 13:03:49.511521  7638 raft_consensus.cc:2243] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:49.511886  7638 raft_consensus.cc:2272] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:49.566766 17140 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.570279 17066 consensus_peers.cc:597] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.592401  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
W20260709 13:03:49.641813 17140 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.665920 17140 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.666697 17140 consensus_peers.cc:597] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.677754 17066 consensus_peers.cc:597] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.681874 16992 consensus_peers.cc:597] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.689419  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:49.702962 17693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:49.707334 17694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:49.709024  7638 server_base.cc:1061] running on GCE node
W20260709 13:03:49.708020 17697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:49.711774 17140 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:49.713404  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:49.713694  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:49.713892  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602229713880 us; error 0 us; skew 500 ppm
I20260709 13:03:49.714550  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:49.718125  7638 webserver.cc:533] Webserver started at http://127.7.117.129:33137/ using document root <none> and password file <none>
I20260709 13:03:49.718695  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:49.718858  7638 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:03:49.722455  7638 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:03:49.730422 17702 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:49.731339  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:03:49.731747  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "64734f14219348f59d1cf8700416565c"
format_stamp: "Formatted at 2026-07-09 13:03:37 on dist-test-slave-ndzg"
I20260709 13:03:49.732288  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:49.746520  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:49.747665  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:49.749130  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:49.753526 17709 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:03:49.784219 16992 consensus_peers.cc:597] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.793896 17140 consensus_peers.cc:597] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:49.823033 17140 consensus_peers.cc:597] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:50.005836  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:03:50.006261  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.254s	user 0.006s	sys 0.002s
I20260709 13:03:50.006649  7638 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:03:50.011575 17709 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.023846 17709 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.024731 17709 tablet_bootstrap.cc:492] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.025326 17709 ts_tablet_manager.cc:1403] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:03:50.027853 17709 raft_consensus.cc:359] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.028494 17709 raft_consensus.cc:740] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.029374 17709 consensus_queue.cc:260] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.031916 17709 ts_tablet_manager.cc:1434] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:50.032546 17709 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.052134 17709 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.053045 17709 tablet_bootstrap.cc:492] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.053697 17709 ts_tablet_manager.cc:1403] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 13:03:50.055987 17709 raft_consensus.cc:359] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.056962 17709 raft_consensus.cc:740] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.057744 17709 consensus_queue.cc:260] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.059535 17709 ts_tablet_manager.cc:1434] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:50.060309 17709 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.108260 17709 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.109166 17709 tablet_bootstrap.cc:492] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.109789 17709 ts_tablet_manager.cc:1403] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.050s	user 0.036s	sys 0.012s
I20260709 13:03:50.112980 17709 raft_consensus.cc:359] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.113890 17709 raft_consensus.cc:740] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.114651 17709 consensus_queue.cc:260] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.116231 17709 ts_tablet_manager.cc:1434] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:50.116858 17709 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.131783 17709 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.132694 17709 tablet_bootstrap.cc:492] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.133312 17709 ts_tablet_manager.cc:1403] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:03:50.136221 17709 raft_consensus.cc:359] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.136910 17709 raft_consensus.cc:740] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.137709 17709 consensus_queue.cc:260] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.139552 17709 ts_tablet_manager.cc:1434] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:50.140257 17709 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.158138 17709 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.159117 17709 tablet_bootstrap.cc:492] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.159760 17709 ts_tablet_manager.cc:1403] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:03:50.162202 17709 raft_consensus.cc:359] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.162863 17709 raft_consensus.cc:740] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.163542 17709 consensus_queue.cc:260] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.165439 17709 ts_tablet_manager.cc:1434] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:50.166182 17709 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.189630 17709 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.190543 17709 tablet_bootstrap.cc:492] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.191134 17709 ts_tablet_manager.cc:1403] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 13:03:50.193459 17709 raft_consensus.cc:359] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.194103 17709 raft_consensus.cc:740] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.195006 17709 consensus_queue.cc:260] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.196966 17709 ts_tablet_manager.cc:1434] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:50.197728 17709 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.214448 17709 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.215376 17709 tablet_bootstrap.cc:492] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.215999 17709 ts_tablet_manager.cc:1403] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 13:03:50.218487 17709 raft_consensus.cc:359] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.219165 17709 raft_consensus.cc:740] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.219835 17709 consensus_queue.cc:260] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.221936 17709 ts_tablet_manager.cc:1434] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.009s	sys 0.000s
I20260709 13:03:50.222699 17709 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.237011 17709 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.237967 17709 tablet_bootstrap.cc:492] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.238598 17709 ts_tablet_manager.cc:1403] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:03:50.240990 17709 raft_consensus.cc:359] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.241685 17709 raft_consensus.cc:740] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.242317 17709 consensus_queue.cc:260] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.244215 17709 ts_tablet_manager.cc:1434] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:50.244959 17709 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.255721  7638 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:03:50.256143  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.250s	user 0.239s	sys 0.000s
I20260709 13:03:50.260490 17709 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.261930 17709 tablet_bootstrap.cc:492] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.263559 17709 ts_tablet_manager.cc:1403] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:03:50.266140 17709 raft_consensus.cc:359] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.266839 17709 raft_consensus.cc:740] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.267511 17709 consensus_queue.cc:260] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.269534 17709 ts_tablet_manager.cc:1434] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:50.270321 17709 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.284124 17709 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.285339 17709 tablet_bootstrap.cc:492] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.286053 17709 ts_tablet_manager.cc:1403] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.001s
I20260709 13:03:50.288856 17709 raft_consensus.cc:359] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.289619 17709 raft_consensus.cc:740] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.290369 17709 consensus_queue.cc:260] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.292136 17709 ts_tablet_manager.cc:1434] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:50.292865 17709 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.326534 17649 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.327345 17649 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:50.336477 17066 leader_election.cc:336] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:03:50.337735 17649 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.339018 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.340971 17067 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:50.341737 17709 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.342180 17649 raft_consensus.cc:2749] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:50.343150 17709 tablet_bootstrap.cc:492] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.343823 17709 ts_tablet_manager.cc:1403] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.051s	user 0.032s	sys 0.012s
I20260709 13:03:50.346168 17709 raft_consensus.cc:359] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.346714 17709 raft_consensus.cc:740] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.347791 17709 consensus_queue.cc:260] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.350088 17709 ts_tablet_manager.cc:1434] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:50.350786 17709 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.390451 17709 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.391496 17709 tablet_bootstrap.cc:492] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.392233 17709 ts_tablet_manager.cc:1403] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.004s
I20260709 13:03:50.394681 17709 raft_consensus.cc:359] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.395351 17709 raft_consensus.cc:740] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.396150 17709 consensus_queue.cc:260] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.398226 17709 ts_tablet_manager.cc:1434] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:50.399276 17709 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.411494 17709 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.412251 17709 tablet_bootstrap.cc:492] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.412777 17709 ts_tablet_manager.cc:1403] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:03:50.414803 17709 raft_consensus.cc:359] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.415337 17709 raft_consensus.cc:740] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.415980 17709 consensus_queue.cc:260] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.417960 17709 ts_tablet_manager.cc:1434] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:03:50.418627 17709 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.422452  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:38761
I20260709 13:03:50.424878 17781 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:38761 every 8 connection(s)
I20260709 13:03:50.434605 17635 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.435482 17635 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.438529 17635 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.441171 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.442126 17100 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:50.443524 16993 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:50.444875 17695 raft_consensus.cc:2804] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.445420 17695 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.445917 17695 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.449579 17649 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.450157 17649 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.454384 17711 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.455078 17711 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.457871 17709 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.461148 17709 tablet_bootstrap.cc:492] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.463563 17709 ts_tablet_manager.cc:1403] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.012s
I20260709 13:03:50.464268 17556 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.465844 17556 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.468861 17709 raft_consensus.cc:359] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.510139 17171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:50.511979 17709 raft_consensus.cc:740] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.512755 17649 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.513551 17709 consensus_queue.cc:260] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.514648 17556 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.515369 17649 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.519109 17649 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.515647 17709 ts_tablet_manager.cc:1434] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.011s
I20260709 13:03:50.521034 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:50.521268 17649 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.522089 17709 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.523049 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.523454 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.523772 17753 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.518458 17711 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.527822 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.528508 17754 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:50.529928 17066 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
W20260709 13:03:50.530550 16992 leader_election.cc:343] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:50.527833 17140 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:50.532112 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.532207 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.533003 17753 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:50.533404 17021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.535037 17710 raft_consensus.cc:2804] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.537308 17710 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.538093 17710 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.536592 17649 raft_consensus.cc:2804] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.538698 17649 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.539104 17649 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.536449 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.546684 17460 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:50.535190 17066 leader_election.cc:343] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:50.547199 17460 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.547605 17066 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:50.549208 17460 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.548744 17066 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7
I20260709 13:03:50.552052 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.552758 17689 raft_consensus.cc:2749] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:50.553332 17753 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.553508 17689 raft_consensus.cc:2804] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.554103 17689 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.554205 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.554723 17689 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.558974 17649 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.560427 17695 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.561005 17140 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:50.562558 17460 raft_consensus.cc:2804] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.562929 17460 raft_consensus.cc:493] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.562759 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:50.563484 17756 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.564442 17649 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.566776 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:50.569459 17460 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.576100 17556 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.578701 17556 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.587386 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.589217 17753 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.588263 17635 raft_consensus.cc:493] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.592394 17140 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:50.590919 17635 raft_consensus.cc:515] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.594676 17649 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.595247 17649 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.597681 17794 raft_consensus.cc:2804] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.598140 17794 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.598482 17672 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.599184 17672 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.601408 17672 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.601563 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.602072 17755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.598487 17794 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.605387 17755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.588264 17793 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:50.606680 17066 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:50.609455 17066 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:50.614179 17460 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.606099 17756 raft_consensus.cc:2468] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:03:50.622470 17635 leader_election.cc:290] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.622890 17789 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.623553 17789 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.624476 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.618620 17711 raft_consensus.cc:2749] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:50.625881 17789 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.624867 17556 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.627022 17710 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.627501 17689 raft_consensus.cc:515] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.627941 17695 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.628360 17556 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.630439 17100 raft_consensus.cc:2393] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:50.608963 17649 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.629732 17460 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.651144 17066 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:50.623574 17635 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:50.654505 16992 leader_election.cc:343] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:50.618939 17793 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.669303 17460 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:50.667101 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 46603.3us
I20260709 13:03:50.676258 17460 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.637789 17556 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.662910 17649 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.669986 17710 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.662166 17635 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.679792 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
W20260709 13:03:50.697583 17066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 46577.5us
I20260709 13:03:50.691581 17711 raft_consensus.cc:2804] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:50.692466 17689 leader_election.cc:290] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.689057 17649 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.705828 17797 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.705973 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:50.706380 17797 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.707026 17101 raft_consensus.cc:2393] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:50.707259 17649 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.708632 17792 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.709210 17792 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.705142 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.711004 17753 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:50.696357 17756 raft_consensus.cc:3060] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.711829 17794 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.713950 17755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:50.701753 17711 raft_consensus.cc:697] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:50.716289 17635 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.717586 17789 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.721779 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:50.721899 17789 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.724246 17789 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.717262 17711 consensus_queue.cc:237] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.705021 17710 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.722237 17709 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:50.729633 17460 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.717509 17672 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.740720 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.752512 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec250363e734059ae59f362df78c792" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.719537 17796 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.708642 17797 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:50.725208 17799 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.739586 17672 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.740442 17689 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.732676 17649 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.760211 17709 tablet_bootstrap.cc:492] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:50.760416 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eccff1f699a4e4387998ef0842e7d33" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:50.761606 17175 raft_consensus.cc:2393] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cc950e241a640e0912a73ed9309b1e5 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:50.761826 17709 ts_tablet_manager.cc:1403] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.240s	user 0.054s	sys 0.053s
I20260709 13:03:50.747251 17789 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.747212 17710 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.773545 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.774169 17710 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.774246 17753 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:50.771036 17798 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.770828 17789 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.771351 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:50.770874 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.779423 17096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.779718 17099 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:50.780179 17066 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:50.780442 17022 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:50.782167 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.783298 17801 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.783486 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.783934 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.764086 17799 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.776093 17024 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.778470 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:50.774695 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:50.786548 17175 raft_consensus.cc:2468] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:50.785212 16992 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:50.778800 17794 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.783880 17801 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.788195 16993 leader_election.cc:304] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:50.770211 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:50.800697 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.804183 17709 raft_consensus.cc:359] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.817024 17709 raft_consensus.cc:740] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:50.795616 17747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.774286 17556 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.766412 17689 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.818996 17709 consensus_queue.cc:260] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:50.800349 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.791882 17097 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.768365 17649 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:50.852094 17140 leader_election.cc:343] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:50.805843 17792 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.767825 17796 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.862521 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:50.798687 17799 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.796275 17789 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:50.803180 17755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:50.792691 17801 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:50.846032 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 57909.1us
I20260709 13:03:50.856943 17709 ts_tablet_manager.cc:1434] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.011s
I20260709 13:03:50.858207 17067 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:50.840404 17635 raft_consensus.cc:2749] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:50.863590 17175 raft_consensus.cc:2393] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:50.828914 17025 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.818040 17797 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.841634 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.805410 17793 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:50.815810 17805 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.849758 17747 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.799824 17748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:50.841049 17756 raft_consensus.cc:2468] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:50.797483 17809 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.870182 17709 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:50.871259 16993 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:50.795413 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.850679 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.797868 17749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.797420 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:50.905331 17067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 47210.7us
I20260709 13:03:50.789469 17798 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.871616 17755 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.876096 17099 raft_consensus.cc:2468] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:50.876156 17792 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:50.967398 16992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 182199us
I20260709 13:03:50.908070 17800 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:51.065965 16992 leader_election.cc:343] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.066201 17798 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:51.066946 16992 leader_election.cc:343] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:51.067731 16992 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.068208 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:50.876881 17797 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:50.933454 17805 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.069411 17025 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:50.981896 17794 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.985561 17710 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:50.992640 17748 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.997721 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:50.994642 17753 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:50.995028 17749 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
W20260709 13:03:51.020720 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 149561us
I20260709 13:03:51.009914 17672 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.021543 17635 raft_consensus.cc:2804] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:50.998880 17809 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.001457 17792 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.033981 17817 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.037781 17097 raft_consensus.cc:2468] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 2.
I20260709 13:03:51.037977 17806 raft_consensus.cc:2749] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.029057 17796 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.116523 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.087797 17710 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.117313 17752 raft_consensus.cc:2468] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:51.045395 17811 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.058523 17782 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42727
I20260709 13:03:51.065845 17800 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:50.937175 17802 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.122546 17802 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.123003 17802 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.029433 17689 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.070901 17805 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.089843 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.126911 17024 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.070902 17797 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:51.107321 17141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 63691.5us
I20260709 13:03:51.094569 17801 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.093487 17635 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.108302 17792 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.108251 17817 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.110069 17809 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.075544 17794 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.113466 17100 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.114095 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.116189 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.086341 17798 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.124225 17021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.124804 17811 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.162796 17794 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.163163 17811 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.146884 17635 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.170567 17649 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.171183 17710 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.172031 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.172742 17750 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:51.173722 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.174361 17750 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:51.174906 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.147032 17096 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:51.184397 17755 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.185806 17800 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.188360 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.190675 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.191627 17020 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:51.193027 16992 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:51.193600 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.194200 16992 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:51.194779 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.195429 17750 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:51.196379 17750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.156263 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.197350 17175 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:51.156805 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.198163 17022 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.199010 17805 raft_consensus.cc:2804] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.199542 17805 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.157224 17140 leader_election.cc:304] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:03:51.157534 17021 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.200012 17805 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.157028 17798 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:51.200639 17140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 43494.8us
I20260709 13:03:51.201349 17140 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.201601 17027 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:51.201881 17460 raft_consensus.cc:2804] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:51.202178 17140 leader_election.cc:343] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.202411 17460 raft_consensus.cc:697] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.120906 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.195276 16992 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
W20260709 13:03:51.205413 17066 leader_election.cc:343] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.207571 17066 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:03:51.126600 17782 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:51.203554 17101 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.202816 17140 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.175209 17748 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
W20260709 13:03:51.212821 17066 leader_election.cc:343] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.212318 17798 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:51.215164 17799 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.203235 17460 consensus_queue.cc:237] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.215731 17799 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.216290 17799 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.218168 17815 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.218606 17815 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.218941 17815 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.213142 17140 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.202839 17820 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.224382 17820 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.224776 17820 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.225716 17827 raft_consensus.cc:2804] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.226150 17827 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:51.226459 16992 leader_election.cc:343] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.228551 17067 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.229224 17818 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.229789 17818 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.229903 17067 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.230170 17818 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.230839 17805 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.231948 17711 raft_consensus.cc:2804] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.232580 17711 raft_consensus.cc:493] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.232942 17711 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.233888 17689 raft_consensus.cc:2804] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.234287 17805 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.234405 17635 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.238035 17025 raft_consensus.cc:2393] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:51.238449 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.226468 17827 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.235802 17689 raft_consensus.cc:493] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:03:51.243467 17140 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.248488 17635 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.249385 17795 raft_consensus.cc:2804] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.252624 17795 raft_consensus.cc:493] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.244241 17689 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.253021 17795 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:51.255303 16992 leader_election.cc:343] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.255757 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.215543 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.256480 17754 raft_consensus.cc:2468] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.256654 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.233968 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:03:51.258420 16992 leader_election.cc:343] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.259572 17747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.260232 17747 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.240792 17817 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.262341 17748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.264325 17795 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.266474 17795 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:03:51.272750 17066 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:51.273507 17066 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:51.274081 17066 leader_election.cc:343] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.278749 17751 raft_consensus.cc:1275] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.283715 16848 catalog_manager.cc:5697] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.275807 17140 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.292878 17782 heartbeater.cc:507] Master 127.7.117.190:42727 requested a full tablet report, sending...
I20260709 13:03:51.293314 17022 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.297744 17828 raft_consensus.cc:2804] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.298271 17828 raft_consensus.cc:697] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.298969 17711 raft_consensus.cc:515] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.299113 17828 consensus_queue.cc:237] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.302138 17709 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:51.293885 17140 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
W20260709 13:03:51.302852 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 174739us
I20260709 13:03:51.303339 17140 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.306797 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:51.307499 17101 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.315011 17709 tablet_bootstrap.cc:492] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:51.316131 17709 ts_tablet_manager.cc:1403] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.448s	user 0.047s	sys 0.043s
I20260709 13:03:51.305440 17795 raft_consensus.cc:2804] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.318508 17795 raft_consensus.cc:697] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.319375 17795 consensus_queue.cc:237] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.322466 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:51.322714 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.323132 17098 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.323249 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:51.324999 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.325425 17808 raft_consensus.cc:2804] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.325567 17753 raft_consensus.cc:3060] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.325899 17808 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.326385 17808 raft_consensus.cc:3060] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.302249 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.328579 17022 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.311076 17817 consensus_queue.cc:1048] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:03:51.304101 17747 raft_consensus.cc:2468] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 2.
I20260709 13:03:51.333246 17748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accaf859802f485d9a8e81e692ccdb50" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.303666 16993 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.334353 17827 raft_consensus.cc:515] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.334929 17753 raft_consensus.cc:2468] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.335232 17635 raft_consensus.cc:2804] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.335671 17635 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.336242 17635 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.337105 17799 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.339138 17799 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.340332 17802 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:03:51.344770 17066 leader_election.cc:343] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.345130 16993 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.345722 17067 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.346940 17817 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.347309 17817 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.347666 17817 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.285043 17818 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.349901 16993 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.351118 17797 raft_consensus.cc:2804] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.351768 17797 raft_consensus.cc:697] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:51.354285 17747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.355012 17747 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.356621 17689 raft_consensus.cc:515] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.352841 17797 consensus_queue.cc:237] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.361457 17635 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.361968 17815 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.354753 16992 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:51.306540 17711 leader_election.cc:290] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.366443 17813 raft_consensus.cc:2804] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.367102 17813 raft_consensus.cc:697] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:51.367013 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.368387 17813 consensus_queue.cc:237] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.378448 17635 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:51.380736 17801 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.381414 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:03:51.390677 17140 leader_election.cc:343] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:51.391386 17140 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.397393 17140 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.345094 17827 leader_election.cc:290] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.403025 17141 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.406401 17098 raft_consensus.cc:2393] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:51.403914 17709 raft_consensus.cc:359] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.408414 17709 raft_consensus.cc:740] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:51.409332 17709 consensus_queue.cc:260] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.405526 16993 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.373631 17808 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.424033 17141 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.411765 17748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.426255 17709 ts_tablet_manager.cc:1434] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.001s
I20260709 13:03:51.427064 17709 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:51.428043 17824 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.428462 17824 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.428826 17824 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.428936 17808 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.413744 17556 raft_consensus.cc:2749] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.430616 17747 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 2.
I20260709 13:03:51.432760 17689 leader_election.cc:290] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.402128 17820 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.435895 17820 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:51.436088 17818 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.436587 17755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.437196 17748 raft_consensus.cc:2393] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.437464 17755 raft_consensus.cc:2393] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:51.418154 17802 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:03:51.437888 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.438135 17815 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.438504 17023 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.443060 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.443727 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.444088 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.444382 17754 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.437917 17819 raft_consensus.cc:2804] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.446285 17819 raft_consensus.cc:697] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.446578 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.447407 17172 raft_consensus.cc:2393] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.447196 17819 consensus_queue.cc:237] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.425881 17827 raft_consensus.cc:2749] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.445606 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.449503 17174 raft_consensus.cc:2393] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.450871 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.451498 17024 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:51.453668 17027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:51.453651 17754 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.431025 17175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.456060 17175 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 1.
I20260709 13:03:51.460812 16992 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:51.461184 17141 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.465859 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d970e7113e5440181d65961915967ad" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.422222 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.467000 17827 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.467029 17096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.467602 17827 raft_consensus.cc:697] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:51.468068 17096 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.468554 17827 consensus_queue.cc:237] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.478950 17141 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.474498 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.471514 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.470597 17174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.472807 17710 raft_consensus.cc:2804] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.488011 17710 raft_consensus.cc:493] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.488469 17710 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.489909 17023 raft_consensus.cc:2468] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:03:51.490562 17747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb98a7c474ea41c3a428ce29909fc94f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.483284 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.492863 17098 raft_consensus.cc:2393] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:51.430617 17022 raft_consensus.cc:2468] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
W20260709 13:03:51.469969 17066 leader_election.cc:343] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.494279 17751 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.494560 17789 raft_consensus.cc:2764] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:03:51.495379 17096 raft_consensus.cc:2468] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 2.
I20260709 13:03:51.497032 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.497968 17754 raft_consensus.cc:2393] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cc950e241a640e0912a73ed9309b1e5 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.503925 17140 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.505158 17793 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.505602 17793 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.505985 17793 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.510052 17020 raft_consensus.cc:3060] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.501773 17174 raft_consensus.cc:2393] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.441190 17817 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.512072 16993 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.521221 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.524087 17172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.528250 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.522648 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.522689 17173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:03:51.533619 17173 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:03:51.531251 17067 leader_election.cc:304] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.536226 17801 raft_consensus.cc:2804] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.536635 17067 leader_election.cc:304] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.536753 17801 raft_consensus.cc:697] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:51.528887 17170 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.537950 17801 consensus_queue.cc:237] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.539876 17689 raft_consensus.cc:2804] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.540464 17689 raft_consensus.cc:697] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:51.525583 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.543675 17710 raft_consensus.cc:515] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.545269 17817 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.546586 17754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
W20260709 13:03:51.548252 17066 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.548777 17173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.549669 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.549726 17689 consensus_queue.cc:237] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.550671 17169 raft_consensus.cc:2393] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.548343 17824 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.551632 17167 raft_consensus.cc:2393] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cc950e241a640e0912a73ed9309b1e5 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.542412 17098 raft_consensus.cc:2393] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
W20260709 13:03:51.556237 16992 leader_election.cc:343] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.526774 17795 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.561406 16993 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.562309 17813 raft_consensus.cc:2804] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.562408 17170 raft_consensus.cc:2468] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in term 2.
I20260709 13:03:51.562884 17813 raft_consensus.cc:697] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:51.563819 17813 consensus_queue.cc:237] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.566083 16993 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.568526 17829 raft_consensus.cc:2749] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.569960 16993 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.557595 17789 raft_consensus.cc:2749] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.560902 17710 leader_election.cc:290] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.557893 17824 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.575754 16846 catalog_manager.cc:5697] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.587543 16846 catalog_manager.cc:5697] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.576176 17829 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.589685 17829 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.590070 17829 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.595453 17020 raft_consensus.cc:1275] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.596668 17753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.597354 17829 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.597688 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.602533 16846 catalog_manager.cc:5697] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.579007 17141 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:51.611506 17710 raft_consensus.cc:2749] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.612041 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.613999 17710 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:51.614670 17710 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.615331 17710 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.573848 17795 raft_consensus.cc:697] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.629940 17793 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.632742 17793 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:51.632926 17140 leader_election.cc:343] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.634002 17751 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.639290 17067 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.634099 16845 catalog_manager.cc:5697] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.641160 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.650180 17098 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.650398 17067 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.657361 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.658509 17100 raft_consensus.cc:2393] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate 1cc950e241a640e0912a73ed9309b1e5 in this term.
I20260709 13:03:51.606757 17817 consensus_queue.cc:1048] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:03:51.635274 17829 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.670032 17096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4035d55f3884082a4ba26e608bfdd6b" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:51.673341 17096 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.675532 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.675875 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.676704 17795 consensus_queue.cc:237] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.659483 17815 raft_consensus.cc:2804] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:03:51.687812 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.132:38527, user_credentials={real_user=slave}} blocked reactor thread for 113621us
I20260709 13:03:51.672315 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.689486 16993 leader_election.cc:304] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.690081 17168 raft_consensus.cc:2393] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cc950e241a640e0912a73ed9309b1e5 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.690623 16993 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.690824 17818 raft_consensus.cc:2749] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.691877 17829 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.692952 16993 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.686169 17815 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:03:51.694799 17672 raft_consensus.cc:2804] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:03:51.695281 17140 leader_election.cc:343] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:51.697517 16992 leader_election.cc:343] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.699437 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afe5ed1a667143e0869e8f06a2a27686" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.695995 17140 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:51.698381 17067 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.705613 17101 raft_consensus.cc:1275] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.695487 17672 raft_consensus.cc:697] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Leader. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:03:51.705724 17811 raft_consensus.cc:2749] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.707620 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.713629 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.714560 17022 raft_consensus.cc:2393] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:51.723723 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.695307 17815 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.700263 17020 raft_consensus.cc:2393] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
W20260709 13:03:51.728719 17140 leader_election.cc:343] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.731884 17140 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.729606 17796 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.735527 17796 raft_consensus.cc:697] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.731386 17098 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.736676 17800 raft_consensus.cc:2749] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.734223 17141 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:51.733891 17672 consensus_queue.cc:237] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.736397 17796 consensus_queue.cc:237] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.741539 17820 raft_consensus.cc:2749] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.751626 17141 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.753610 17460 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.754142 17460 raft_consensus.cc:697] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.755342 17807 consensus_queue.cc:1048] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:03:51.746155 17793 raft_consensus.cc:2749] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.760553 17460 consensus_queue.cc:237] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.787914 17020 raft_consensus.cc:1275] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.799669 17067 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:51.800853 17811 raft_consensus.cc:2749] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.805860 17753 raft_consensus.cc:1275] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.807338 17815 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.808732 17067 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.778170 17096 raft_consensus.cc:2468] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:51.810679 17711 raft_consensus.cc:2749] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.794881 17710 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:51.812376 17170 raft_consensus.cc:2393] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in current term 2: Already voted for candidate 8267993b082f449891e464bc4dee65f7 in this term.
I20260709 13:03:51.812774 17807 consensus_queue.cc:1048] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.819721 17795 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:51.823114 16993 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:03:51.834807 17709 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:51.832118 17141 leader_election.cc:304] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:51.828866 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:51.824891 17818 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.840454 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:51.819176 17710 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:03:51.846131 17140 leader_election.cc:343] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.859261 17709 tablet_bootstrap.cc:492] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:51.860404 17709 ts_tablet_manager.cc:1403] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.433s	user 0.034s	sys 0.007s
I20260709 13:03:51.861232 17807 raft_consensus.cc:2804] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:51.861838 17807 raft_consensus.cc:697] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:51.862679 17807 consensus_queue.cc:237] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:51.864073 17815 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:51.863694 17709 raft_consensus.cc:359] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.864682 17709 raft_consensus.cc:740] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:51.847806 17020 raft_consensus.cc:2393] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8267993b082f449891e464bc4dee65f7 in current term 2: Already voted for candidate ece68ef4a10b4f60b2f20b1e5c0a1ab4 in this term.
I20260709 13:03:51.868734 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
W20260709 13:03:51.874228 16992 consensus_peers.cc:597] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:51.874756 16848 catalog_manager.cc:5697] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.876148 17709 consensus_queue.cc:260] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:51.877807 16847 ts_manager.cc:194] Re-registered known tserver with Master: 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:03:51.886690 17066 leader_election.cc:343] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:51.897766 16848 catalog_manager.cc:5697] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.904352 17096 raft_consensus.cc:1275] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:51.905514 16992 consensus_peers.cc:597] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:51.907296 17709 ts_tablet_manager.cc:1434] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:03:51.909138 17709 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:51.911023 17168 raft_consensus.cc:1275] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.912909 17751 raft_consensus.cc:1275] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.914258 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:03:51.917179 17818 consensus_queue.cc:1048] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.917464 17846 consensus_queue.cc:1048] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.919149 17846 consensus_queue.cc:1048] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:51.919368 16992 consensus_peers.cc:597] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:51.930871 17140 consensus_peers.cc:597] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:51.929379 17756 raft_consensus.cc:1275] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.944887 17822 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:51.946900 17020 raft_consensus.cc:1275] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.956676 17753 raft_consensus.cc:1275] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.959663 17460 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:51.962170 17819 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.953470 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "855f8184b9834eeb8c9957f6a2df61f4" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:03:51.965698 17170 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.969172 16846 catalog_manager.cc:5697] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.980072 17756 raft_consensus.cc:3060] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:51.980868 16846 catalog_manager.cc:5697] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.985526 17753 raft_consensus.cc:1275] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.985304 16846 catalog_manager.cc:5697] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.987267 16846 catalog_manager.cc:5697] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.992914 17710 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.995708 17819 consensus_queue.cc:1048] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.000694 16846 catalog_manager.cc:5697] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.997516 17753 raft_consensus.cc:1275] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.013873 17818 consensus_queue.cc:1048] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.040148 17025 raft_consensus.cc:1275] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.042884 17811 consensus_queue.cc:1048] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.056232 17023 raft_consensus.cc:1275] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:52.062638 17140 consensus_peers.cc:597] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.082860 17756 raft_consensus.cc:1275] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.084182 17170 raft_consensus.cc:2468] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:03:52.084573 17811 consensus_queue.cc:1048] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:03:52.090369 17098 raft_consensus.cc:1275] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.095880 17797 consensus_queue.cc:1048] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:03:52.111217 17067 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:03:52.122797 17820 consensus_queue.cc:1048] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.119305 17800 raft_consensus.cc:2804] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:52.125473 17800 raft_consensus.cc:697] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:03:52.126379 17800 consensus_queue.cc:237] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:03:52.132098 17066 consensus_peers.cc:597] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:52.134028 16992 consensus_peers.cc:597] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:52.139034 17066 consensus_peers.cc:597] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:52.141036 17140 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.152170 17026 raft_consensus.cc:1275] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:52.150990 16992 consensus_peers.cc:597] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.156607 17815 consensus_queue.cc:1048] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.139328 17755 raft_consensus.cc:1275] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.203435 16848 catalog_manager.cc:5697] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.210067 17556 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.228667 16848 catalog_manager.cc:5697] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.231036 16848 catalog_manager.cc:5697] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.233383 16848 catalog_manager.cc:5697] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.252499 17756 raft_consensus.cc:1275] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:52.290830 17066 consensus_peers.cc:597] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:52.291577 17066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 76861.7us
I20260709 13:03:52.303696 17709 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:03:52.318192 16992 consensus_peers.cc:597] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.322914 17101 raft_consensus.cc:1275] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.337556 17709 tablet_bootstrap.cc:492] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.338694 17709 ts_tablet_manager.cc:1403] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.430s	user 0.013s	sys 0.015s
W20260709 13:03:52.343092 17140 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.344573 17024 raft_consensus.cc:1275] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.324672 17822 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:03:52.334173 17803 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:03:52.362247 17099 raft_consensus.cc:1275] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.374615 17709 raft_consensus.cc:359] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:52.375695 17709 raft_consensus.cc:740] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.366791 17096 raft_consensus.cc:1275] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.387415 17807 consensus_queue.cc:1048] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.364007 17167 raft_consensus.cc:1275] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer ece68ef4a10b4f60b2f20b1e5c0a1ab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.376564 17709 consensus_queue.cc:260] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:52.395987 17094 raft_consensus.cc:1275] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:52.397516 17066 consensus_peers.cc:597] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.404572 17796 consensus_queue.cc:1048] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.408367 17880 consensus_queue.cc:1048] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:52.412536 17140 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.418077 17709 ts_tablet_manager.cc:1434] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:03:52.443506 17709 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
W20260709 13:03:52.461750 17140 consensus_peers.cc:597] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:52.462553 17140 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:52.463267 17140 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.463733 17807 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:52.464883 16992 consensus_peers.cc:597] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:52.490749 16992 consensus_peers.cc:597] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.496037 17798 consensus_queue.cc:1048] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:52.558657 17140 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:52.579241 17066 consensus_peers.cc:597] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.583077 16845 catalog_manager.cc:5697] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.611318 16847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47040
I20260709 13:03:52.617133 17709 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:03:52.625922 17140 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.628980 17782 heartbeater.cc:499] Master 127.7.117.190:42727 was elected leader, sending a full tablet report...
I20260709 13:03:52.632762 17709 tablet_bootstrap.cc:492] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.633597 17709 ts_tablet_manager.cc:1403] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.196s	user 0.037s	sys 0.019s
W20260709 13:03:52.641554 17140 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.645493 17709 raft_consensus.cc:359] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.646314 17709 raft_consensus.cc:740] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.647006 17709 consensus_queue.cc:260] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.649830 17709 ts_tablet_manager.cc:1434] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:03:52.654428 17709 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
W20260709 13:03:52.683072 16992 consensus_peers.cc:597] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.732513 17168 raft_consensus.cc:1275] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.735149 17711 consensus_queue.cc:1048] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.779323 17709 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.780273 17709 tablet_bootstrap.cc:492] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.780857 17709 ts_tablet_manager.cc:1403] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.127s	user 0.025s	sys 0.011s
I20260709 13:03:52.783327 17709 raft_consensus.cc:359] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.783987 17709 raft_consensus.cc:740] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.784647 17709 consensus_queue.cc:260] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.786702 17709 ts_tablet_manager.cc:1434] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:52.787714 17709 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
W20260709 13:03:52.795627 16992 consensus_peers.cc:597] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:52.816727 17709 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.817880 17709 tablet_bootstrap.cc:492] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.818531 17709 ts_tablet_manager.cc:1403] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.008s
I20260709 13:03:52.821081 17709 raft_consensus.cc:359] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:52.821800 17709 raft_consensus.cc:740] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.822475 17709 consensus_queue.cc:260] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:52.824462 17709 ts_tablet_manager.cc:1434] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:52.825213 17709 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:52.850157 17709 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.851190 17709 tablet_bootstrap.cc:492] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.851915 17709 ts_tablet_manager.cc:1403] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.007s
I20260709 13:03:52.854547 17709 raft_consensus.cc:359] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:52.855262 17709 raft_consensus.cc:740] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.855926 17709 consensus_queue.cc:260] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:52.858201 17709 ts_tablet_manager.cc:1434] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:03:52.859383 17709 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:52.875679 17709 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.876636 17709 tablet_bootstrap.cc:492] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.877391 17709 ts_tablet_manager.cc:1403] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:03:52.880095 17709 raft_consensus.cc:359] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.880767 17709 raft_consensus.cc:740] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.881443 17709 consensus_queue.cc:260] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.883494 17709 ts_tablet_manager.cc:1434] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:52.884589 17709 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:52.915966 17709 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.916962 17709 tablet_bootstrap.cc:492] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.917701 17709 ts_tablet_manager.cc:1403] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.004s
I20260709 13:03:52.920361 17709 raft_consensus.cc:359] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:52.921069 17709 raft_consensus.cc:740] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.921923 17709 consensus_queue.cc:260] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:52.924166 17709 ts_tablet_manager.cc:1434] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:52.927114 17709 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:52.943439 17709 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:52.944314 17709 tablet_bootstrap.cc:492] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:52.944856 17709 ts_tablet_manager.cc:1403] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.001s
I20260709 13:03:52.946720 17709 raft_consensus.cc:359] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.947229 17709 raft_consensus.cc:740] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.947728 17709 consensus_queue.cc:260] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:52.949333 17709 ts_tablet_manager.cc:1434] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.003s
I20260709 13:03:52.950484 17709 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:52.970119 17752 raft_consensus.cc:3060] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.030846 17709 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.031903 17709 tablet_bootstrap.cc:492] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.032603 17709 ts_tablet_manager.cc:1403] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.082s	user 0.017s	sys 0.032s
I20260709 13:03:53.035100 17709 raft_consensus.cc:359] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.035777 17709 raft_consensus.cc:740] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.036455 17709 consensus_queue.cc:260] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.038425 17709 ts_tablet_manager.cc:1434] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:03:53.039240 17709 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.053011 17709 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.053995 17709 tablet_bootstrap.cc:492] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.054592 17709 ts_tablet_manager.cc:1403] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:03:53.056339 17709 raft_consensus.cc:359] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.056885 17709 raft_consensus.cc:740] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.057502 17709 consensus_queue.cc:260] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.059017 17709 ts_tablet_manager.cc:1434] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:03:53.059640 17709 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.083292 17709 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.085340 17709 tablet_bootstrap.cc:492] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.085922 17709 ts_tablet_manager.cc:1403] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 13:03:53.087863 17709 raft_consensus.cc:359] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.088357 17709 raft_consensus.cc:740] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.088973 17709 consensus_queue.cc:260] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.090911 17709 ts_tablet_manager.cc:1434] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.092084 17709 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.105687 17752 raft_consensus.cc:3060] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.105815 17709 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.106809 17709 tablet_bootstrap.cc:492] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.107383 17709 ts_tablet_manager.cc:1403] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.007s
I20260709 13:03:53.109797 17709 raft_consensus.cc:359] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.110460 17709 raft_consensus.cc:740] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.111035 17709 consensus_queue.cc:260] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.112792 17709 ts_tablet_manager.cc:1434] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.113590 17709 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.128834 17709 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.129760 17709 tablet_bootstrap.cc:492] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.130448 17709 ts_tablet_manager.cc:1403] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.001s
I20260709 13:03:53.132789 17709 raft_consensus.cc:359] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.133467 17709 raft_consensus.cc:740] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.134105 17709 consensus_queue.cc:260] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.135993 17709 ts_tablet_manager.cc:1434] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:53.136900 17709 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.152469 17709 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.153265 17709 tablet_bootstrap.cc:492] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.153859 17709 ts_tablet_manager.cc:1403] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:03:53.155632 17709 raft_consensus.cc:359] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.156265 17709 raft_consensus.cc:740] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.156944 17709 consensus_queue.cc:260] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.159775 17709 ts_tablet_manager.cc:1434] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:53.160704 17709 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.178120 17709 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.178974 17709 tablet_bootstrap.cc:492] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.179481 17709 ts_tablet_manager.cc:1403] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:03:53.181982 17709 raft_consensus.cc:359] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.182446 17709 raft_consensus.cc:740] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.182991 17709 consensus_queue.cc:260] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.184377 17709 ts_tablet_manager.cc:1434] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.185437 17709 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.198271 17709 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.199242 17709 tablet_bootstrap.cc:492] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.199826 17709 ts_tablet_manager.cc:1403] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 13:03:53.202145 17709 raft_consensus.cc:359] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.202749 17709 raft_consensus.cc:740] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.203392 17709 consensus_queue.cc:260] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.205538 17709 ts_tablet_manager.cc:1434] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:03:53.206884 17709 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.229497 17709 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.230461 17709 tablet_bootstrap.cc:492] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.231110 17709 ts_tablet_manager.cc:1403] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.012s
I20260709 13:03:53.233685 17709 raft_consensus.cc:359] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.234369 17709 raft_consensus.cc:740] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.234999 17709 consensus_queue.cc:260] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.237015 17709 ts_tablet_manager.cc:1434] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:53.241542 17709 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.278426 17709 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.279359 17709 tablet_bootstrap.cc:492] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.280524 17709 ts_tablet_manager.cc:1403] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.011s
I20260709 13:03:53.283036 17709 raft_consensus.cc:359] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.283730 17709 raft_consensus.cc:740] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.284430 17709 consensus_queue.cc:260] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.286688 17709 ts_tablet_manager.cc:1434] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:53.288910 17709 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.302498 17709 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.303277 17709 tablet_bootstrap.cc:492] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.303885 17709 ts_tablet_manager.cc:1403] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:03:53.306145 17709 raft_consensus.cc:359] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.306690 17709 raft_consensus.cc:740] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.307271 17709 consensus_queue.cc:260] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.309026 17709 ts_tablet_manager.cc:1434] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.309903 17709 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.321921 17709 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.322652 17709 tablet_bootstrap.cc:492] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.323176 17709 ts_tablet_manager.cc:1403] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:03:53.325654 17709 raft_consensus.cc:359] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.326160 17709 raft_consensus.cc:740] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.326854 17709 consensus_queue.cc:260] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.328645 17709 ts_tablet_manager.cc:1434] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.329602 17709 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.342957 17709 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.343847 17709 tablet_bootstrap.cc:492] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.344502 17709 ts_tablet_manager.cc:1403] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 13:03:53.347014 17709 raft_consensus.cc:359] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.347668 17709 raft_consensus.cc:740] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.348417 17709 consensus_queue.cc:260] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.350203 17709 ts_tablet_manager.cc:1434] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:03:53.351639 17709 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.352425 17807 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.352938 17807 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.354974 17807 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:53.356995 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:03:53.357728 17097 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:53.359134 17141 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:53.359949 17822 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:53.360322 17822 raft_consensus.cc:493] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.360646 17822 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.360698 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:53.361937 17751 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:53.368168 17822 raft_consensus.cc:515] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.370976 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:03:53.371167 17822 leader_election.cc:290] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:03:53.371577 17097 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.372864 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca37e8a20c2447983a81703b8706d51" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:53.373562 17751 raft_consensus.cc:3060] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.376552 17709 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.377401 17097 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:03:53.377668 17709 tablet_bootstrap.cc:492] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.378520 17709 ts_tablet_manager.cc:1403] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 13:03:53.378602 17141 leader_election.cc:304] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:53.379581 17822 raft_consensus.cc:2804] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:53.380066 17822 raft_consensus.cc:697] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:53.380911 17822 consensus_queue.cc:237] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.381857 17751 raft_consensus.cc:2468] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:03:53.382261 17709 raft_consensus.cc:359] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.383040 17709 raft_consensus.cc:740] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.383792 17709 consensus_queue.cc:260] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.385862 17709 ts_tablet_manager.cc:1434] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:53.386758 17709 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.397797 16847 catalog_manager.cc:5697] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 3, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:53.417752 17709 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.418815 17709 tablet_bootstrap.cc:492] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.419821 17709 ts_tablet_manager.cc:1403] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.000s
I20260709 13:03:53.422562 17709 raft_consensus.cc:359] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.423295 17709 raft_consensus.cc:740] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.426404 17751 raft_consensus.cc:3060] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.427443 17709 consensus_queue.cc:260] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.464439 17709 ts_tablet_manager.cc:1434] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.002s
I20260709 13:03:53.467195 17709 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.470623 17751 raft_consensus.cc:3060] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.481339 17752 raft_consensus.cc:3060] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.501434 17709 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.508435 17709 tablet_bootstrap.cc:492] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.509142 17709 ts_tablet_manager.cc:1403] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.012s
I20260709 13:03:53.511670 17709 raft_consensus.cc:359] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.512326 17709 raft_consensus.cc:740] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.512938 17709 consensus_queue.cc:260] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.514581 17709 ts_tablet_manager.cc:1434] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.515323 17709 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.537696 17709 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.538774 17709 tablet_bootstrap.cc:492] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.539515 17709 ts_tablet_manager.cc:1403] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 13:03:53.542086 17709 raft_consensus.cc:359] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.542768 17709 raft_consensus.cc:740] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.543515 17709 consensus_queue.cc:260] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:03:53.546425 17709 ts_tablet_manager.cc:1434] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:53.562177 17709 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.580051 17709 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.580951 17709 tablet_bootstrap.cc:492] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.581554 17709 ts_tablet_manager.cc:1403] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:03:53.583851 17709 raft_consensus.cc:359] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.584499 17709 raft_consensus.cc:740] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.585165 17709 consensus_queue.cc:260] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.587229 17709 ts_tablet_manager.cc:1434] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:53.588353 17709 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.602947 17709 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.603881 17709 tablet_bootstrap.cc:492] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.604564 17709 ts_tablet_manager.cc:1403] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.003s
I20260709 13:03:53.606648 17709 raft_consensus.cc:359] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.607156 17709 raft_consensus.cc:740] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.607874 17709 consensus_queue.cc:260] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:03:53.609746 17709 ts_tablet_manager.cc:1434] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:03:53.610603 17709 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Bootstrap starting.
I20260709 13:03:53.627125 17709 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.627952 17709 tablet_bootstrap.cc:492] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Bootstrap complete.
I20260709 13:03:53.628561 17709 ts_tablet_manager.cc:1403] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:03:53.630839 17709 raft_consensus.cc:359] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.631388 17709 raft_consensus.cc:740] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64734f14219348f59d1cf8700416565c, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.632040 17709 consensus_queue.cc:260] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.633947 17709 ts_tablet_manager.cc:1434] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:53.675073 17752 raft_consensus.cc:3060] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.765611 17752 raft_consensus.cc:1275] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:53.766808 17803 consensus_queue.cc:1048] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:53.779471 17098 raft_consensus.cc:1275] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:53.781260 17803 consensus_queue.cc:1048] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:53.882334 17751 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:53.917737 17933 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.918133 17933 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.919797 17933 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:53.920464 17751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:03:53.920652 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:03:53.921110 17751 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:03:53.921212 17024 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:03:53.922123 17140 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:03:53.922832 17933 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:53.923184 17933 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.923447 17933 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.929927 17933 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.932006 17933 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Requested vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:03:53.932592 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c"
I20260709 13:03:53.932798 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:03:53.933192 17752 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.933389 17024 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:53.938548 17024 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:03:53.939435 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:03:53.939451 17752 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:03:53.940285 17803 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:53.940752 17803 raft_consensus.cc:697] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:03:53.941474 17803 consensus_queue.cc:237] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:03:53.950529 16845 catalog_manager.cc:5697] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 3, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:54.332528 17023 raft_consensus.cc:1275] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:54.334208 17933 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:54.347932 17751 raft_consensus.cc:1275] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:54.349519 17933 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:57.872349 17066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 48356.2us
W20260709 13:03:57.893483 17140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 51161.2us
W20260709 13:03:59.816216 17141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 459656us
I20260709 13:04:00.507880  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:00.562188 17025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de"
dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:36198
I20260709 13:04:00.563154 17025 raft_consensus.cc:606] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:00.575928 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:00.576581 17167 raft_consensus.cc:606] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:00.579077 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:00.579578 17167 raft_consensus.cc:606] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:04:00.581892 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:00.582474 17167 raft_consensus.cc:606] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:00.584609 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:00.585078 17167 raft_consensus.cc:606] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:00.587287 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:00.587779 17167 raft_consensus.cc:606] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:00.588996  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:00.590171  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:00.700588 17048 debug-util.cc:398] Leaking SignalData structure 0x7b0800214060 after lost signal to thread 7641
W20260709 13:04:00.701751 17048 debug-util.cc:398] Leaking SignalData structure 0x7b0800128260 after lost signal to thread 16884
W20260709 13:04:00.703557 17048 debug-util.cc:398] Leaking SignalData structure 0x7b0800225ca0 after lost signal to thread 17125
W20260709 13:04:00.705413 17048 debug-util.cc:398] Leaking SignalData structure 0x7b08002b36c0 after lost signal to thread 17781
I20260709 13:04:00.726604 17960 raft_consensus.cc:993] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.727298 17973 raft_consensus.cc:1081] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.729000 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34328
I20260709 13:04:00.729560 17756 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.729924 17756 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.736384 17756 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:00.742005 17756 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:00.761772 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:00.762699 17167 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.772779 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed34ea43214b00a76e89f8b09249de" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:00.773479 17025 raft_consensus.cc:3055] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:00.773788 17025 raft_consensus.cc:740] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:04:00.774611 17025 consensus_queue.cc:260] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:00.776991 17025 raft_consensus.cc:3060] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.793467 17025 raft_consensus.cc:2468] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:00.797511 17706 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:00.798241 17975 raft_consensus.cc:993] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.798838 17975 raft_consensus.cc:1081] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.800863 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:00.801775 17756 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.802285 17756 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.805949 17167 raft_consensus.cc:2468] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:00.812829 17995 raft_consensus.cc:2804] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:00.821612 17756 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:00.825273 17995 raft_consensus.cc:697] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:00.826645 17995 consensus_queue.cc:237] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:00.836619 17986 raft_consensus.cc:993] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: : Instructing follower 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:00.837183 17986 raft_consensus.cc:1081] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:00.844233 17097 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e"
dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:59236
I20260709 13:04:00.844776 17097 raft_consensus.cc:493] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.845134 17097 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.845249 17756 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:00.854090 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:00.854700 17168 raft_consensus.cc:3055] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:00.854969 17168 raft_consensus.cc:740] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:00.855779 17168 consensus_queue.cc:260] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:00.857084 17168 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.863099 16845 catalog_manager.cc:5697] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.868777 17168 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:00.872776 17706 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:00.873867 17995 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:00.874389 17995 raft_consensus.cc:697] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:00.875473 17995 consensus_queue.cc:237] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:00.879537 17097 raft_consensus.cc:515] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:00.885773 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:00.886404 17026 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.887169 17097 leader_election.cc:290] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:00.887724 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b5b7c58e054ac6ac9e77e29ed86e5e" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:00.888310 17168 raft_consensus.cc:3055] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:00.888571 17168 raft_consensus.cc:740] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:00.890980 17168 consensus_queue.cc:260] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:00.892077 17168 raft_consensus.cc:3060] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.959170 17975 raft_consensus.cc:993] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.959878 17975 raft_consensus.cc:1081] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.965657 17751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:00.966297 17751 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.966689 17751 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.976799 17168 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:00.981246 17975 raft_consensus.cc:993] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.981904 17975 raft_consensus.cc:1081] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:00.984010 17755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:00.984763 17755 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.985183 17755 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:00.981155 17026 raft_consensus.cc:2468] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:00.990530 17067 leader_election.cc:304] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:00.991524 17962 raft_consensus.cc:2804] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:00.991986 17962 raft_consensus.cc:697] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:00.992906 17962 consensus_queue.cc:237] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:00.982592 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:00.997388 17751 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.013712 17101 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.030535 17966 raft_consensus.cc:993] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:01.031313 17966 raft_consensus.cc:1081] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:01.010851 17755 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.035063 17755 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:01.037045 17097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:01.037940 17097 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:01.053208 17066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 52893.9us
I20260709 13:04:01.054998 17097 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.058320 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:01.058632 17706 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:04:01.058955 17168 raft_consensus.cc:3055] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:01.059319 17168 raft_consensus.cc:740] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:01.059548 17995 raft_consensus.cc:2804] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:01.060076 17168 consensus_queue.cc:260] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.061405 17168 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.060109 17995 raft_consensus.cc:697] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:01.062727 17995 consensus_queue.cc:237] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.066713 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:01.067279 17756 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:01.067623 17756 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.071762 17751 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:01.073117 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:01.073767 17167 raft_consensus.cc:3055] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:01.074074 17167 raft_consensus.cc:740] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:01.074831 17167 consensus_queue.cc:260] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.078804 17167 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.083030 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:01.083659 17098 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.087280 17168 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.091307 17101 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:01.102993 17756 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:01.108966 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:01.109849 17168 raft_consensus.cc:3055] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:01.110330 17168 raft_consensus.cc:740] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:01.113024 17168 consensus_queue.cc:260] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:01.119252 17756 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:01.120126 17168 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.116998 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2819a664db491aa04d5c6ec4c13708" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:01.125875 17020 raft_consensus.cc:3060] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.132062 17168 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:01.133705 17706 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:01.135566 16845 catalog_manager.cc:5697] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.135689 17996 raft_consensus.cc:2804] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:01.139632 17020 raft_consensus.cc:2468] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:01.141877 17167 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.142655 17098 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.148775 17706 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:01.155472 16847 catalog_manager.cc:5697] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:01.157809 16847 catalog_manager.cc:5697] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:01.161087 17995 raft_consensus.cc:2804] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:01.165902 17141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 71389.1us
I20260709 13:04:01.166278 17995 raft_consensus.cc:697] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:01.167304 17995 consensus_queue.cc:237] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.169872 17020 raft_consensus.cc:1240] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Rejecting Update request from peer 8267993b082f449891e464bc4dee65f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:01.171669 17996 raft_consensus.cc:697] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:01.174909 17996 consensus_queue.cc:237] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:01.198524 16847 catalog_manager.cc:5697] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.199857 16847 catalog_manager.cc:5697] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:01.211211 17020 raft_consensus.cc:1275] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.212672 17996 consensus_queue.cc:1048] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.239012 17168 raft_consensus.cc:1275] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.242916 17995 consensus_queue.cc:1048] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.424271 17168 raft_consensus.cc:1275] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:01.425603 17995 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.435585 17101 raft_consensus.cc:1275] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:01.437083 18012 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.472512 17168 raft_consensus.cc:1275] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.476289 17962 consensus_queue.cc:1048] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.484630 17168 raft_consensus.cc:1275] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.488567 18012 consensus_queue.cc:1048] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.503145 17101 raft_consensus.cc:1275] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.508809 18011 consensus_queue.cc:1048] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:01.531538 17020 raft_consensus.cc:1275] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.533689 17981 consensus_queue.cc:1048] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:01.580363 17168 raft_consensus.cc:1275] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.582353 18012 consensus_queue.cc:1048] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.591080  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:01.596735 17020 raft_consensus.cc:1275] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:01.598634 18012 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.610880 17101 raft_consensus.cc:1275] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:01.614248 17995 consensus_queue.cc:1048] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:01.693470 17168 raft_consensus.cc:1275] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:01.695811 18012 consensus_queue.cc:1048] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.725131  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:01.777204 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:01.777930 17168 raft_consensus.cc:606] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:01.780702 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:01.781324 17168 raft_consensus.cc:606] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:01.785174 17026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "391757a5f74549289d0b597a84df6740"
dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
mode: GRACEFUL
new_leader_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:36198
I20260709 13:04:01.785727 17026 raft_consensus.cc:606] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Received request to transfer leadership to TS 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:04:01.789252 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:01.789996 17168 raft_consensus.cc:606] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:04:01.792789 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:01.793382 17167 raft_consensus.cc:606] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:01.795665 17167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:01.796157 17167 raft_consensus.cc:606] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:01.797485  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:01.798880  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:01.867865 18005 raft_consensus.cc:993] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:01.868552 17987 raft_consensus.cc:1081] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:01.870213 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:01.870893 17756 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:01.871295 17756 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.879230 17756 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.881198 17756 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:01.884475 17096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:01.885157 17096 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.884745 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:01.886036 17167 raft_consensus.cc:3055] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:01.886348 17167 raft_consensus.cc:740] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:01.887212 17167 consensus_queue.cc:260] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.888485 17167 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.894447 17096 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.897266 17706 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:04:01.901561 17996 raft_consensus.cc:2804] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:01.902107 17996 raft_consensus.cc:697] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:01.903088 17996 consensus_queue.cc:237] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:01.938126 17167 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:01.939837 16847 catalog_manager.cc:5697] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:01.975713 17706 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 73215.1us
I20260709 13:04:02.094484 18010 raft_consensus.cc:993] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.095049 17986 raft_consensus.cc:1081] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.096954 17751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:02.097488 17751 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.097873 17751 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.104341 18005 raft_consensus.cc:993] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.105808 18005 raft_consensus.cc:1081] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.107748 17755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:02.108346 17755 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.108711 17755 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:02.112579 17751 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.115401 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.116020 17167 raft_consensus.cc:3055] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.116333 17167 raft_consensus.cc:740] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:02.117182 17167 consensus_queue.cc:260] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.118819 17167 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.118887 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100511a7ba6744e08fb2dec95a371354" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:02.119491 17026 raft_consensus.cc:3060] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.124550 17751 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:02.126546 17167 raft_consensus.cc:2468] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:02.127921 17706 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:02.117911 17755 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:02.129046 18012 raft_consensus.cc:2804] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.129568 18012 raft_consensus.cc:697] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:02.130712 17755 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:02.130446 18012 consensus_queue.cc:237] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.131856 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.132484 17168 raft_consensus.cc:3055] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:02.132769 17168 raft_consensus.cc:740] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:02.133600 17168 consensus_queue.cc:260] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:02.135046 17168 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:02.135869 18005 raft_consensus.cc:993] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:02.136934 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:02.137547 17098 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:02.139793 18010 raft_consensus.cc:1081] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:02.144222 17096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834"
dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:59236
I20260709 13:04:02.144812 17096 raft_consensus.cc:493] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.145159 17096 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.144351 17098 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:02.145608 17026 raft_consensus.cc:2468] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:02.146646 17706 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:04:02.147601 18031 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:02.148082 18031 raft_consensus.cc:697] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:02.148947 18031 consensus_queue.cc:237] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:02.154045 17096 raft_consensus.cc:515] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.156734 17096 leader_election.cc:290] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:02.161636 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.162117 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633fd2f819d45d893ebcdb54511d834" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:02.162245 17170 raft_consensus.cc:3055] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.162798 17026 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.162807 17170 raft_consensus.cc:740] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:02.163909 17170 consensus_queue.cc:260] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.165262 17170 raft_consensus.cc:3060] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.168437 17168 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:02.168267 16847 catalog_manager.cc:5697] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.171041 17026 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:02.172411 17067 leader_election.cc:304] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:02.172777 17170 raft_consensus.cc:2468] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:02.173377 18008 raft_consensus.cc:2804] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.173931 18008 raft_consensus.cc:697] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:02.174791 18008 consensus_queue.cc:237] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.185909 16847 catalog_manager.cc:5697] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.190198 16845 catalog_manager.cc:5697] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.235365 17987 raft_consensus.cc:993] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.235903 17987 raft_consensus.cc:1081] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.237495 17755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:02.238013 17755 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.238339 17755 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.246414 17960 raft_consensus.cc:993] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: : Instructing follower 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:02.247138 17992 raft_consensus.cc:1081] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Signalling peer 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:02.248838 17096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "391757a5f74549289d0b597a84df6740"
dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:59244
I20260709 13:04:02.249447 17096 raft_consensus.cc:493] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.249817 17096 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.250454 17755 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.254329 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:02.254904 17026 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.255512 17170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a02330036da4716bc0da7d166bdea11" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.256131 17170 raft_consensus.cc:3055] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.256438 17170 raft_consensus.cc:740] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:02.257298 17170 consensus_queue.cc:260] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.258375 17170 raft_consensus.cc:3060] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.258535 17096 raft_consensus.cc:515] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.262243 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:02.262326 17096 leader_election.cc:290] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:02.262907 17020 raft_consensus.cc:3055] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.262779 17026 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:02.263422 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "391757a5f74549289d0b597a84df6740" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.263588 17020 raft_consensus.cc:740] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:04:02.264019 17168 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.263553 17755 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:02.264555 17706 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:02.264606 17020 consensus_queue.cc:260] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.267324 17020 raft_consensus.cc:3060] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.266204 18012 raft_consensus.cc:2804] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.269541 18012 raft_consensus.cc:697] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:02.270792 18012 consensus_queue.cc:237] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.276167 17020 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:02.277485 17067 leader_election.cc:304] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:02.278368 18008 raft_consensus.cc:2804] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.278826 18008 raft_consensus.cc:697] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:02.279690 18008 consensus_queue.cc:237] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:02.283386 17170 raft_consensus.cc:2468] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:02.289906 17168 raft_consensus.cc:2468] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:02.300269 16845 catalog_manager.cc:5697] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.315057 16847 catalog_manager.cc:5697] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.328377 17096 raft_consensus.cc:1275] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.329900 18012 consensus_queue.cc:1048] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.349614 17168 raft_consensus.cc:1275] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.356331 18031 consensus_queue.cc:1048] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.535416 17098 raft_consensus.cc:1275] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:02.536568 18031 consensus_queue.cc:1048] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:02.564579 17020 raft_consensus.cc:1275] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.570766 18008 consensus_queue.cc:1048] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.581321 17168 raft_consensus.cc:1275] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:02.583384 18031 consensus_queue.cc:1048] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:02.597090 17170 raft_consensus.cc:1275] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.612136 17981 consensus_queue.cc:1048] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:02.673550 17020 raft_consensus.cc:1275] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.675017 18031 consensus_queue.cc:1048] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.687914 17168 raft_consensus.cc:1275] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.689652 18012 consensus_queue.cc:1048] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.712541 17020 raft_consensus.cc:1275] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.714141 18004 consensus_queue.cc:1048] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.735183 17168 raft_consensus.cc:1275] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.737147 17981 consensus_queue.cc:1048] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.788165 17168 raft_consensus.cc:1275] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.790036 18031 consensus_queue.cc:1048] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.797569 17020 raft_consensus.cc:1275] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:02.799166 18012 consensus_queue.cc:1048] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:02.799764  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:02.882711  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:02.943761 17020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd"
dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:36198
I20260709 13:04:02.944394 17020 raft_consensus.cc:606] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:02.947011 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:02.947458 17168 raft_consensus.cc:606] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:02.950642 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:02.951143 17168 raft_consensus.cc:606] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 1cc950e241a640e0912a73ed9309b1e5
I20260709 13:04:02.952895 17987 raft_consensus.cc:993] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.953442 17987 raft_consensus.cc:1081] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:02.954988 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:02.954994 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c455336bc164af1af57c6148b97d553"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:02.955636 17756 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.955658 17168 raft_consensus.cc:606] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:02.956127 17756 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:02.958052 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:02.958487 17168 raft_consensus.cc:606] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:02.960845 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:02.961194 17168 raft_consensus.cc:606] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
I20260709 13:04:02.962210  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:02.963397  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:02.964325 17756 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.966404 17756 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 4 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:02.967648 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:02.967842 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abd5bac63004970887f5d75e88e9a0f" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:02.968324 17168 raft_consensus.cc:3055] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:02.968765 17168 raft_consensus.cc:740] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:02.968353 17020 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:02.969816 17168 consensus_queue.cc:260] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.971019 17168 raft_consensus.cc:3060] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:02.976061 17020 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 4.
I20260709 13:04:02.977398 17706 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4; no voters: 
I20260709 13:04:02.978232 18031 raft_consensus.cc:2804] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:02.978516 17168 raft_consensus.cc:2468] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 4.
I20260709 13:04:02.978729 18031 raft_consensus.cc:697] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 4 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:02.979784 18031 consensus_queue.cc:237] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:02.991793 16847 catalog_manager.cc:5697] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 3 to 4, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 4 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.013960 17986 raft_consensus.cc:993] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:03.016351 18005 raft_consensus.cc:1081] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 1cc950e241a640e0912a73ed9309b1e5 to start an election
I20260709 13:04:03.019186 17096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6"
dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
 from {username='slave'} at 127.0.0.1:59236
I20260709 13:04:03.019706 17096 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.019924 17096 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.024688 17096 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:03.026158 17096 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:03.026889 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:03.027478 17020 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.027349 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ed6b9a59b31479aaa96d38781de77d6" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:03.027980 17168 raft_consensus.cc:3055] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.028214 17168 raft_consensus.cc:740] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:03.028916 17168 consensus_queue.cc:260] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:03.029825 17168 raft_consensus.cc:3060] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.036702 17168 raft_consensus.cc:2468] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:03.037997 17067 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:03.038522 17020 raft_consensus.cc:2468] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
I20260709 13:04:03.038781 17981 raft_consensus.cc:2804] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.039702 17981 raft_consensus.cc:697] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:03.040653 17981 consensus_queue.cc:237] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:03.053839 16847 catalog_manager.cc:5697] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.072371 17992 raft_consensus.cc:993] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.072908 17960 raft_consensus.cc:1081] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.074573 17751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34328
I20260709 13:04:03.075105 17751 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.075423 17751 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.096771 17751 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:03.109338 17098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:03.109340 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ae68a5d259b476daae7a5665770a0cd" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:03.110116 17098 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.110209 17020 raft_consensus.cc:3055] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:03.110702 17020 raft_consensus.cc:740] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ece68ef4a10b4f60b2f20b1e5c0a1ab4, State: Running, Role: LEADER
I20260709 13:04:03.111445 17020 consensus_queue.cc:260] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:03.112488 17020 raft_consensus.cc:3060] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.116819 17098 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:03.118135 17706 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c; no voters: 
I20260709 13:04:03.118662 17020 raft_consensus.cc:2468] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:03.118978 17996 raft_consensus.cc:2804] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:03.123535 17996 raft_consensus.cc:697] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:03.124459 17996 consensus_queue.cc:237] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:03.109130 17751 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:03.159722 16847 catalog_manager.cc:5697] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.432705 18005 raft_consensus.cc:993] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.433212 18005 raft_consensus.cc:1081] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.435235 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:03.435756 17756 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.436118 17756 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.437014 17986 raft_consensus.cc:993] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.437547 17987 raft_consensus.cc:1081] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.439160 17751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c455336bc164af1af57c6148b97d553"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:03.439671 17751 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.439980 17751 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.442546 17756 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.445152 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:03.445688 17168 raft_consensus.cc:3055] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:03.445940 17168 raft_consensus.cc:740] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:03.446630 17168 consensus_queue.cc:260] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.447628 17168 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.446659 17751 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.450202 17751 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:03.452608 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:03.452946 17756 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:03.453244 17167 raft_consensus.cc:3055] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.453564 17167 raft_consensus.cc:740] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:03.454262 17167 consensus_queue.cc:260] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.455265 17167 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.456686 17020 raft_consensus.cc:1275] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.457806 17168 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:03.459116 17706 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:03.460027 17996 raft_consensus.cc:2804] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:03.459865 18004 consensus_queue.cc:1048] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.461581 17996 raft_consensus.cc:697] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:03.462522 17996 consensus_queue.cc:237] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.463105 17025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdaa7a451fbb426d868eef2a6abd8849" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:03.463728 17025 raft_consensus.cc:3060] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:03.462719 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c455336bc164af1af57c6148b97d553" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:03.472335 17020 raft_consensus.cc:3060] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.492007 17023 raft_consensus.cc:1275] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 4 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:03.494314 18031 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:03.494952 17987 raft_consensus.cc:993] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.495675 17987 raft_consensus.cc:1081] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:03.462754 17167 raft_consensus.cc:2468] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:03.497444 17756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:03.497979 17756 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.498413 17756 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.499825 16847 catalog_manager.cc:5697] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.504649 17706 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:03.509897 18012 raft_consensus.cc:2804] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.510432 18012 raft_consensus.cc:697] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:03.510619 17025 raft_consensus.cc:2468] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:03.511372 18012 consensus_queue.cc:237] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.526114 17756 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.530040 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:03.530692 17168 raft_consensus.cc:3055] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.531061 17168 raft_consensus.cc:740] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:03.531850 17168 consensus_queue.cc:260] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.532961 17168 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.533608 17756 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:03.537227 17170 raft_consensus.cc:1275] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:03.500952 17020 raft_consensus.cc:2468] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:03.542515 17169 raft_consensus.cc:1275] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.548192 17996 consensus_queue.cc:1048] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:03.548273 18004 consensus_queue.cc:1048] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.585870 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:03.586468 17020 raft_consensus.cc:3060] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.590529 17023 raft_consensus.cc:1275] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:03.604662 17020 raft_consensus.cc:2468] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:03.608325 17168 raft_consensus.cc:2468] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 2.
I20260709 13:04:03.620769 17996 consensus_queue.cc:1048] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:03.628501 16845 catalog_manager.cc:5697] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.690217 17706 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:03.691020 17098 raft_consensus.cc:1275] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:03.713824 17996 raft_consensus.cc:2804] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.732688 18060 consensus_queue.cc:1048] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:03.735720 17996 raft_consensus.cc:697] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:03.760185 17996 consensus_queue.cc:237] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:03.791872 16845 catalog_manager.cc:5697] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 1 to 2, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 2 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:03.818424 17706 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:42149, user_credentials={real_user=slave}} blocked reactor thread for 60431.6us
W20260709 13:04:03.889536 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 65518.4us
I20260709 13:04:03.921619 17170 raft_consensus.cc:1275] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:03.931492 17026 raft_consensus.cc:1275] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:03.933856 18031 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:03.935313 18031 consensus_queue.cc:1048] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:04:03.966888  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:04.025341 17169 raft_consensus.cc:1275] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.033710 18060 consensus_queue.cc:1048] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:04.252662 17025 raft_consensus.cc:1275] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.288106 17023 raft_consensus.cc:1275] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.949254 18084 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:04.954541 18084 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:04.956732 18084 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:04.967238 17169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.002s	sys 0.012s
W20260709 13:04:04.967696 17169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.003s	sys 0.012s
I20260709 13:04:04.971956 17022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:04.992789 18094 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:04.993338 18094 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:04.995402 18094 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:05.002949 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:05.011152 18084 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:05.011785 18084 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:05.012799 17756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:04:05.014271 18084 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:05.016397 18031 consensus_queue.cc:1048] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:05.022404 17026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.007s	sys 0.005s
I20260709 13:04:05.040724 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:04:05.042171 17026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.008s	sys 0.006s
W20260709 13:04:05.041417 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 44780.6us
I20260709 13:04:05.016768 17066 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:04:05.018101 18031 consensus_queue.cc:1048] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:04:05.060369 18083 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:05.060941 18083 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:05.063503 18083 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:05.067602 18091 raft_consensus.cc:2749] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:05.055473 17026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36200 (request call id 41) took 1103 ms. Trace:
I20260709 13:04:05.068687 17026 rpcz_store.cc:276] 0709 13:04:03.939748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.939894 (+   146us) service_pool.cc:224] Handling call
0709 13:04:03.940178 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.940529 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.940559 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.941042 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.042559 (+1101517us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:05.042588 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.042671 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.042709 (+    38us) raft_consensus.cc:1685] finished
0709 13:04:05.042720 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.042826 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8399,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":2168,"thread_start_us":1100980,"threads_started":1,"wal-append.queue_time_us":94444,"wal-append.run_cpu_time_us":1869,"wal-append.run_wall_time_us":1001888}
I20260709 13:04:05.073630 17986 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:05.074195 17986 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:05.076864 17986 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:05.082506 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:05.091987 17024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a40bdee1b5c3486b94412af27867b738" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:05.095074 17173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eae2b6552a54c4bbaff1296163ac17b" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:05.096586 18084 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:05.107043 17167 raft_consensus.cc:1275] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:05.118818  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:05.109462 18084 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:04:05.152997 16993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 107995us
I20260709 13:04:05.159701 17749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
W20260709 13:04:05.164386 17706 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.130:42149, user_credentials={real_user=slave}} blocked reactor thread for 55122.5us
I20260709 13:04:05.175634 17706 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.183797 18084 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:05.199252 17026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:05.201403 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33c172d9d8e34d8b86a2d40dd97cfc0d" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:05.217905 17141 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.218748 18105 consensus_queue.cc:1048] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:05.219112 17141 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.220286 18096 raft_consensus.cc:2749] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:05.220949 18096 raft_consensus.cc:2749] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:05.196713 18031 raft_consensus.cc:2749] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:05.234525 16992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:38761, user_credentials={real_user=slave}} blocked reactor thread for 61569.1us
I20260709 13:04:05.243959 18097 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.244530 18097 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:05.280755 18097 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:05.282071 17752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "64734f14219348f59d1cf8700416565c" is_pre_election: true
I20260709 13:04:05.292567 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a2edef1b2a42bca289c34851377b55" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:05.293354 17168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5"
dest_uuid: "8267993b082f449891e464bc4dee65f7"
mode: GRACEFUL
new_leader_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:51554
I20260709 13:04:05.293942 17168 raft_consensus.cc:606] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Received request to transfer leadership to TS 64734f14219348f59d1cf8700416565c
W20260709 13:04:05.294476 17706 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:43115, user_credentials={real_user=slave}} blocked reactor thread for 80747us
I20260709 13:04:05.295181 17706 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.297153 18100 raft_consensus.cc:2749] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:05.298398 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:05.303362 17023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "355dd8ce521a412c8b28a5988e30bf7e" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" is_pre_election: true
I20260709 13:04:05.297981 17141 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.304594  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:04:05.308847  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:05.313900 18097 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:05.321384 17706 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64734f14219348f59d1cf8700416565c; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:05.323943 18098 raft_consensus.cc:2749] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:05.555938 17122 debug-util.cc:398] Leaking SignalData structure 0x7b0800305780 after lost signal to thread 7641
W20260709 13:04:05.556921 17122 debug-util.cc:398] Leaking SignalData structure 0x7b08002636e0 after lost signal to thread 16884
W20260709 13:04:05.558372 17122 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5780 after lost signal to thread 17051
W20260709 13:04:05.559691 17122 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8940 after lost signal to thread 17125
W20260709 13:04:05.561007 17122 debug-util.cc:398] Leaking SignalData structure 0x7b080025c6e0 after lost signal to thread 17199
W20260709 13:04:05.562405 17122 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1220 after lost signal to thread 17781
I20260709 13:04:05.595885 18005 raft_consensus.cc:993] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: : Instructing follower 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:05.596477 18005 raft_consensus.cc:1081] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Signalling peer 64734f14219348f59d1cf8700416565c to start an election
I20260709 13:04:05.608130 17752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5"
dest_uuid: "64734f14219348f59d1cf8700416565c"
 from {username='slave'} at 127.0.0.1:34318
I20260709 13:04:05.608700 17752 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.609055 17752 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.618549 17752 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:05.627954 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:05.628625 17168 raft_consensus.cc:3055] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:05.628934 17168 raft_consensus.cc:740] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:05.629832 17168 consensus_queue.cc:260] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:05.631008 17168 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.632735 17752 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:05.634629 17020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8756dd718c74c838e90911ac1fb65e5" candidate_uuid: "64734f14219348f59d1cf8700416565c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4"
I20260709 13:04:05.635263 17020 raft_consensus.cc:3060] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.640249 17168 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:05.641741 17706 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:05.642688 18111 raft_consensus.cc:2804] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:05.642863 17020 raft_consensus.cc:2468] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64734f14219348f59d1cf8700416565c in term 3.
I20260709 13:04:05.643273 18111 raft_consensus.cc:697] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Becoming Leader. State: Replica: 64734f14219348f59d1cf8700416565c, State: Running, Role: LEADER
I20260709 13:04:05.645471 18111 consensus_queue.cc:237] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:05.680152 16845 catalog_manager.cc:5697] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c reported cstate change: term changed from 2 to 3, leader changed from 8267993b082f449891e464bc4dee65f7 (127.7.117.132) to 64734f14219348f59d1cf8700416565c (127.7.117.129). New cstate: current_term: 3 leader_uuid: "64734f14219348f59d1cf8700416565c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:06.083762 17167 raft_consensus.cc:1275] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:06.085134 18100 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.136888 17020 raft_consensus.cc:1275] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Refusing update from remote peer 64734f14219348f59d1cf8700416565c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:06.141744 18111 consensus_queue.cc:1048] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.309901  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:06.399663  7638 tablet_server.cc:179] TabletServer@127.7.117.129:38761 shutting down...
W20260709 13:04:06.411283 17066 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 58 similar messages]
W20260709 13:04:06.430812 17066 consensus_peers.cc:597] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.445637 16992 consensus_peers.cc:597] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.460532 16992 consensus_peers.cc:597] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.466277 17140 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.468796 17140 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.485513 16992 consensus_peers.cc:597] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.486605 16992 consensus_peers.cc:597] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.513343 16992 consensus_peers.cc:597] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.514472  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:06.515501  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.516531  7638 raft_consensus.cc:2243] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.516958  7638 raft_consensus.cc:2272] T 8fd44ab20c334b99964d344244e650ad P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.520880  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.521737  7638 raft_consensus.cc:2243] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.522590  7638 raft_consensus.cc:2272] T a9a2edef1b2a42bca289c34851377b55 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.525163  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.525884  7638 raft_consensus.cc:2243] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.526283  7638 raft_consensus.cc:2272] T 117045952bb14176abd739933fa51259 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.528704  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.529397  7638 raft_consensus.cc:2243] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.529896  7638 raft_consensus.cc:2272] T eb98a7c474ea41c3a428ce29909fc94f P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.565716  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.571815  7638 raft_consensus.cc:2243] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.572366  7638 raft_consensus.cc:2272] T f0c2977ecbd943b3bbffc28c746df99f P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.582598  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.583444  7638 raft_consensus.cc:2243] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.584013  7638 raft_consensus.cc:2272] T 7eae2b6552a54c4bbaff1296163ac17b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.590819 17066 consensus_peers.cc:597] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.600108 16992 consensus_peers.cc:597] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.605466 16992 consensus_peers.cc:597] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.603519 17066 consensus_peers.cc:597] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.603636  7638 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:06.612005 17140 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.613055  7638 raft_consensus.cc:2243] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.613641  7638 raft_consensus.cc:2272] T 8f8e87277d714ded843585986eae389e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.614418 17140 consensus_peers.cc:597] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.619201  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.622146  7638 raft_consensus.cc:2243] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.623317  7638 raft_consensus.cc:2272] T 16ada941539640f58a0f2ba47d6c1954 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.625097 17066 consensus_peers.cc:597] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.626434  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.627322  7638 raft_consensus.cc:2243] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.628243  7638 raft_consensus.cc:2272] T 6a02330036da4716bc0da7d166bdea11 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.631294  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.632148  7638 raft_consensus.cc:2243] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.632697  7638 raft_consensus.cc:2272] T a40bdee1b5c3486b94412af27867b738 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.635751  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.636622  7638 raft_consensus.cc:2243] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.637130  7638 raft_consensus.cc:2272] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.637255 16992 consensus_peers.cc:597] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.643818  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.644659  7638 raft_consensus.cc:2243] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.645645  7638 raft_consensus.cc:2272] T 2abd5bac63004970887f5d75e88e9a0f P 64734f14219348f59d1cf8700416565c [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.647668 17140 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.648419  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.649343  7638 raft_consensus.cc:2243] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.649880  7638 raft_consensus.cc:2272] T 61ae0ddb123d4d8bb5b071178e546931 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.653005  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.653689  7638 raft_consensus.cc:2243] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.654150  7638 raft_consensus.cc:2272] T ed56f28edb0c443ead4f92414e0a29fa P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.656344  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.657274  7638 raft_consensus.cc:2243] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.658121  7638 raft_consensus.cc:2272] T 100511a7ba6744e08fb2dec95a371354 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.660686  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.661242  7638 raft_consensus.cc:2243] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.662030  7638 raft_consensus.cc:2272] T a97fcf0cdbc04350a9368ac4f9972f72 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.664176  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.664717  7638 raft_consensus.cc:2243] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.665227  7638 raft_consensus.cc:2272] T accaf859802f485d9a8e81e692ccdb50 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.668432  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.669375  7638 raft_consensus.cc:2243] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.669899  7638 raft_consensus.cc:2272] T e5e1d0f0dc3240608259e975a8ea33ec P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.672655  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.673530  7638 raft_consensus.cc:2243] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.674459  7638 raft_consensus.cc:2272] T bf62f622f0524a91bf77aab05b4c9d3f P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.677302  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.678227  7638 raft_consensus.cc:2243] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:06.678442 16992 consensus_peers.cc:597] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.679246  7638 raft_consensus.cc:2272] T 60c79060c3294f4fbe036c1fe48ae9bf P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.681686  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.682399  7638 raft_consensus.cc:2243] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.682868  7638 raft_consensus.cc:2272] T 0eccff1f699a4e4387998ef0842e7d33 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.685426  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.686146  7638 raft_consensus.cc:2243] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.686617  7638 raft_consensus.cc:2272] T 7d970e7113e5440181d65961915967ad P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.689157  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.689930  7638 raft_consensus.cc:2243] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.690490  7638 raft_consensus.cc:2272] T 855f8184b9834eeb8c9957f6a2df61f4 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.692646  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.693311  7638 raft_consensus.cc:2243] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.694206  7638 raft_consensus.cc:2272] T 56ed34ea43214b00a76e89f8b09249de P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.697023  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.697810  7638 raft_consensus.cc:2243] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.698347  7638 raft_consensus.cc:2272] T afe5ed1a667143e0869e8f06a2a27686 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.699957 16992 consensus_peers.cc:597] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.701464  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.702464  7638 raft_consensus.cc:2243] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.703001  7638 raft_consensus.cc:2272] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.706008  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.706861  7638 raft_consensus.cc:2243] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.707371  7638 raft_consensus.cc:2272] T 0395376e4b3d44a0a28a7bb199a21307 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.711185  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.712059  7638 raft_consensus.cc:2243] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.712627  7638 raft_consensus.cc:2272] T 3ca37e8a20c2447983a81703b8706d51 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.715629  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.716501  7638 raft_consensus.cc:2243] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:06.717372 17066 consensus_peers.cc:597] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.717041  7638 raft_consensus.cc:2272] T 5b17e18d23d343e6894f11dc02b2eb75 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.721383  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.722151  7638 raft_consensus.cc:2243] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.724170  7638 raft_consensus.cc:2272] T bd2354d787dc43f58fd361f6b4832507 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.727020  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.727803  7638 raft_consensus.cc:2243] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.728295  7638 raft_consensus.cc:2272] T 9ed0bc1914444d179b36b1253f15c65c P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.731271  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.732048  7638 raft_consensus.cc:2243] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.733006  7638 raft_consensus.cc:2272] T 1ae68a5d259b476daae7a5665770a0cd P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.735670  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.736451  7638 raft_consensus.cc:2243] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.737443  7638 raft_consensus.cc:2272] T 9c455336bc164af1af57c6148b97d553 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.741422  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.742229  7638 raft_consensus.cc:2243] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:06.742367 17066 consensus_peers.cc:597] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.742889  7638 raft_consensus.cc:2272] T d4035d55f3884082a4ba26e608bfdd6b P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.745769  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.748023  7638 raft_consensus.cc:2243] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.748517  7638 raft_consensus.cc:2272] T 162f7d6234e549cc968efac4ee8846ac P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.751458  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.752192  7638 raft_consensus.cc:2243] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.752729  7638 raft_consensus.cc:2272] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.754366 17140 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.755422  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.756191  7638 raft_consensus.cc:2243] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.756659  7638 raft_consensus.cc:2272] T 355dd8ce521a412c8b28a5988e30bf7e P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.759449  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.760306  7638 raft_consensus.cc:2243] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.760794  7638 raft_consensus.cc:2272] T 006cb6681bb949508a3ac253c5a64661 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.763521  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.764223  7638 raft_consensus.cc:2243] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.765174  7638 raft_consensus.cc:2272] T a8756dd718c74c838e90911ac1fb65e5 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.766462 17066 consensus_peers.cc:597] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.768571  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.769449  7638 raft_consensus.cc:2243] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.769975  7638 raft_consensus.cc:2272] T 2ec250363e734059ae59f362df78c792 P 64734f14219348f59d1cf8700416565c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.772686  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.773433  7638 raft_consensus.cc:2243] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.773922  7638 raft_consensus.cc:2272] T de112b6726e84fdca382fa7b9a903c49 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.776772  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.777550  7638 raft_consensus.cc:2243] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.778162  7638 raft_consensus.cc:2272] T c6ce161acdcd435a8f869ae50a6ea162 P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.780936  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.781677  7638 raft_consensus.cc:2243] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.782194  7638 raft_consensus.cc:2272] T e14794cbc74f4fcf8d3872e3463ad58d P 64734f14219348f59d1cf8700416565c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.784883  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.785557  7638 raft_consensus.cc:2243] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.786484  7638 raft_consensus.cc:2272] T cdaa7a451fbb426d868eef2a6abd8849 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.789057  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:06.789814  7638 raft_consensus.cc:2243] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.790812  7638 raft_consensus.cc:2272] T 8d2819a664db491aa04d5c6ec4c13708 P 64734f14219348f59d1cf8700416565c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.791393 17066 consensus_peers.cc:597] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.828514 17140 consensus_peers.cc:597] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.886379 17140 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.899829  7638 tablet_server.cc:196] TabletServer@127.7.117.129:38761 shutdown complete.
W20260709 13:04:06.968598 17140 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.025146 16992 consensus_peers.cc:597] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.027851 17140 consensus_peers.cc:597] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.046702 16992 consensus_peers.cc:597] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.107311  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
W20260709 13:04:07.118558 17067 consensus_peers.cc:597] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.134869 17141 consensus_peers.cc:597] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.168284 17141 consensus_peers.cc:597] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.173542 17067 consensus_peers.cc:597] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.180488 17141 consensus_peers.cc:597] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.193800 17067 consensus_peers.cc:597] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.234218  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:07.235373  7638 tablet_replica.cc:333] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.236367  7638 raft_consensus.cc:2243] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.237447  7638 raft_consensus.cc:2272] T eb98a7c474ea41c3a428ce29909fc94f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.240584  7638 tablet_replica.cc:333] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.242633  7638 raft_consensus.cc:2243] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.243978  7638 raft_consensus.cc:2272] T 5b17e18d23d343e6894f11dc02b2eb75 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.247960  7638 tablet_replica.cc:333] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.248780  7638 raft_consensus.cc:2243] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.249428  7638 raft_consensus.cc:2272] T a9a2edef1b2a42bca289c34851377b55 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.253495  7638 tablet_replica.cc:333] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.254413  7638 raft_consensus.cc:2243] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.255013  7638 raft_consensus.cc:2272] T c6ce161acdcd435a8f869ae50a6ea162 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.258041  7638 tablet_replica.cc:333] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.258968  7638 raft_consensus.cc:2243] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.259575  7638 raft_consensus.cc:2272] T 8d2819a664db491aa04d5c6ec4c13708 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.262351  7638 tablet_replica.cc:333] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.263559  7638 raft_consensus.cc:2243] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.264626  7638 raft_consensus.cc:2272] T 0395376e4b3d44a0a28a7bb199a21307 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.274679  7638 tablet_replica.cc:333] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.276718  7638 raft_consensus.cc:2243] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.290848  7638 raft_consensus.cc:2272] T 391757a5f74549289d0b597a84df6740 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.294239  7638 tablet_replica.cc:333] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.295032  7638 raft_consensus.cc:2243] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.295629  7638 raft_consensus.cc:2272] T 2ec250363e734059ae59f362df78c792 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.301134  7638 tablet_replica.cc:333] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.301836  7638 raft_consensus.cc:2243] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.302341  7638 raft_consensus.cc:2272] T 4ed6b9a59b31479aaa96d38781de77d6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.304800  7638 tablet_replica.cc:333] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.305604  7638 raft_consensus.cc:2243] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.306126  7638 raft_consensus.cc:2272] T 6a02330036da4716bc0da7d166bdea11 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.308823  7638 tablet_replica.cc:333] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.309664  7638 raft_consensus.cc:2243] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.310658  7638 raft_consensus.cc:2272] T a40bdee1b5c3486b94412af27867b738 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.313481  7638 tablet_replica.cc:333] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.314255  7638 raft_consensus.cc:2243] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.314744  7638 raft_consensus.cc:2272] T 4633fd2f819d45d893ebcdb54511d834 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.315277 17067 consensus_peers.cc:597] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.317580  7638 tablet_replica.cc:333] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.318557  7638 raft_consensus.cc:2243] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.319566  7638 raft_consensus.cc:2272] T 7eae2b6552a54c4bbaff1296163ac17b P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.323338  7638 tablet_replica.cc:333] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.324149  7638 raft_consensus.cc:2243] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.324752  7638 raft_consensus.cc:2272] T d3c68d18d16941a9b3197081006477dc P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.328118  7638 tablet_replica.cc:333] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.328986  7638 raft_consensus.cc:2243] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.329617  7638 raft_consensus.cc:2272] T 0c8c7883a65740a5842d54e35490a6e1 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.346462  7638 tablet_replica.cc:333] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
W20260709 13:04:07.348343 17141 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.351986  7638 raft_consensus.cc:2243] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.352672  7638 raft_consensus.cc:2272] T bafbaa58b13148c2b916fe7ee7e234e6 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.353025 17067 consensus_peers.cc:597] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.355863  7638 tablet_replica.cc:333] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.357954  7638 raft_consensus.cc:2243] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.359052  7638 raft_consensus.cc:2272] T 355dd8ce521a412c8b28a5988e30bf7e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.363026  7638 tablet_replica.cc:333] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.364176  7638 raft_consensus.cc:2243] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.365224  7638 raft_consensus.cc:2272] T 006cb6681bb949508a3ac253c5a64661 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.367967  7638 tablet_replica.cc:333] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.368785  7638 raft_consensus.cc:2243] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.369379  7638 raft_consensus.cc:2272] T a8756dd718c74c838e90911ac1fb65e5 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.373633  7638 tablet_replica.cc:333] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.374477  7638 raft_consensus.cc:2243] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.375032  7638 raft_consensus.cc:2272] T 56ed34ea43214b00a76e89f8b09249de P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.376242 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.377918  7638 tablet_replica.cc:333] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.378863  7638 raft_consensus.cc:2243] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.379518  7638 raft_consensus.cc:2272] T afe5ed1a667143e0869e8f06a2a27686 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.382368  7638 tablet_replica.cc:333] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.383224  7638 raft_consensus.cc:2243] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.383831  7638 raft_consensus.cc:2272] T 7d970e7113e5440181d65961915967ad P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.386821  7638 tablet_replica.cc:333] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.387637  7638 raft_consensus.cc:2243] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.388698  7638 raft_consensus.cc:2272] T 33c172d9d8e34d8b86a2d40dd97cfc0d P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.391638  7638 tablet_replica.cc:333] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.392452  7638 raft_consensus.cc:2243] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:04:07.393352 17067 consensus_peers.cc:597] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.393533  7638 raft_consensus.cc:2272] T a9bdaab91b264f3ea1e21283b3c988ac P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.396416  7638 tablet_replica.cc:333] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.397245  7638 raft_consensus.cc:2243] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.397814  7638 raft_consensus.cc:2272] T 89b5b7c58e054ac6ac9e77e29ed86e5e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.399351 17067 consensus_peers.cc:597] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.400579  7638 tablet_replica.cc:333] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.401434  7638 raft_consensus.cc:2243] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.401952  7638 raft_consensus.cc:2272] T 8f8e87277d714ded843585986eae389e P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.404769  7638 tablet_replica.cc:333] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.407014  7638 raft_consensus.cc:2243] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.408257  7638 raft_consensus.cc:2272] T a12c1a9abfa94488a9aefb37cb9ecdf0 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.411573  7638 tablet_replica.cc:333] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.412438  7638 raft_consensus.cc:2243] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.413830  7638 raft_consensus.cc:2272] T 2abd5bac63004970887f5d75e88e9a0f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.421902  7638 tablet_replica.cc:333] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.422781  7638 raft_consensus.cc:2243] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.423887  7638 raft_consensus.cc:2272] T de112b6726e84fdca382fa7b9a903c49 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.426985  7638 tablet_replica.cc:333] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.427879  7638 raft_consensus.cc:2243] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.428998  7638 raft_consensus.cc:2272] T edd447a4120b4d378b710f3f18d0541f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.432525  7638 tablet_replica.cc:333] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
W20260709 13:04:07.432895 17067 consensus_peers.cc:597] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.433394  7638 raft_consensus.cc:2243] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.434031  7638 raft_consensus.cc:2272] T cdaa7a451fbb426d868eef2a6abd8849 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.436653  7638 tablet_replica.cc:333] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.437497  7638 raft_consensus.cc:2243] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.438089  7638 raft_consensus.cc:2272] T 9c455336bc164af1af57c6148b97d553 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.444397  7638 tablet_replica.cc:333] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.445276  7638 raft_consensus.cc:2243] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.445852  7638 raft_consensus.cc:2272] T 1ae68a5d259b476daae7a5665770a0cd P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.448650  7638 tablet_replica.cc:333] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.449476  7638 raft_consensus.cc:2243] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.450012  7638 raft_consensus.cc:2272] T 9ed0bc1914444d179b36b1253f15c65c P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.452682  7638 tablet_replica.cc:333] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.455709  7638 raft_consensus.cc:2243] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.457691  7638 raft_consensus.cc:2272] T 117045952bb14176abd739933fa51259 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.460891  7638 tablet_replica.cc:333] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.461725  7638 raft_consensus.cc:2243] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.462257  7638 raft_consensus.cc:2272] T e797287dc34d42e3838f8f5a3ba68985 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.465427  7638 tablet_replica.cc:333] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.466203  7638 raft_consensus.cc:2243] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.466807  7638 raft_consensus.cc:2272] T afb0f82169da45819893f31678f63f76 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.469615  7638 tablet_replica.cc:333] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.470829  7638 raft_consensus.cc:2243] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.471597  7638 raft_consensus.cc:2272] T 100511a7ba6744e08fb2dec95a371354 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.475801  7638 tablet_replica.cc:333] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.476593  7638 raft_consensus.cc:2243] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.477159  7638 raft_consensus.cc:2272] T e5e1d0f0dc3240608259e975a8ea33ec P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.480888  7638 tablet_replica.cc:333] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.481714  7638 raft_consensus.cc:2243] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.482326  7638 raft_consensus.cc:2272] T accaf859802f485d9a8e81e692ccdb50 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.485183  7638 tablet_replica.cc:333] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.486044  7638 raft_consensus.cc:2243] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.487085  7638 raft_consensus.cc:2272] T f0c2977ecbd943b3bbffc28c746df99f P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.488926 17067 consensus_peers.cc:597] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.489893  7638 tablet_replica.cc:333] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.490711  7638 raft_consensus.cc:2243] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:07.492959  7638 raft_consensus.cc:2272] T c71c71675c2b420aaaad7cd24eef4084 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.495783  7638 tablet_replica.cc:333] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.496510  7638 raft_consensus.cc:2243] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.498814  7638 raft_consensus.cc:2272] T 2db96a4ed139460aa021acdd63b9b529 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:07.502391  7638 tablet_replica.cc:333] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4: stopping tablet replica
I20260709 13:04:07.503090  7638 raft_consensus.cc:2243] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:07.503701  7638 raft_consensus.cc:2272] T 3e944c9ade604302b16ffd9d285523a4 P ece68ef4a10b4f60b2f20b1e5c0a1ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:07.504199 17141 consensus_peers.cc:597] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.505791 17141 consensus_peers.cc:597] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.510975 17067 consensus_peers.cc:597] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.552436 17067 consensus_peers.cc:597] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:07.613987 17141 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.640257 18097 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.640769 18097 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.642933 18097 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.644708 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:04:07.651707 17140 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.652232 17140 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:07.653434 18097 raft_consensus.cc:2749] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.712260 18134 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.712877 18134 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.715201 18134 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.715965 18143 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.716562 18143 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:07.719060 18143 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:07.726347 17140 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:07.734524 17066 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.736358 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:07.737112 17167 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
W20260709 13:04:07.737926 17141 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:07.738346 17141 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:07.738965 18143 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.739645 17067 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:07.740373 18134 raft_consensus.cc:2804] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:07.740724 18134 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.741044 18134 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:07.747666 18134 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.751858 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf62f622f0524a91bf77aab05b4c9d3f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:07.752493 17167 raft_consensus.cc:3060] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:07.758229 17066 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.759018 18134 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.763549 17167 raft_consensus.cc:2468] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 3.
I20260709 13:04:07.769517 17067 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:07.770440 18134 raft_consensus.cc:2804] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:07.771235 18134 raft_consensus.cc:697] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 3 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:07.772132 18134 consensus_queue.cc:237] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.792279 18140 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.792824 18140 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.795143 18140 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:07.803315 17066 leader_election.cc:336] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.803442 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:07.805073 17067 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:04:07.811411 18148 raft_consensus.cc:2749] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.818445 18143 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.819084 18143 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.822587 18143 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:07.826869 17140 leader_election.cc:336] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.805727 16846 catalog_manager.cc:5697] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 2 to 3, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:07.868805 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:07.883886 18143 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.884964 17141 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:07.885205 18143 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:07.888182 18097 raft_consensus.cc:2749] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:07.890050 18149 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.891076 18149 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.897018 18134 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.900701 18134 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.899451 18097 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.904203 18097 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:07.908077 18097 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:07.918264 17140 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.889019 18143 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:07.928220 17141 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:07.930960 18134 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.933730 18134 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.934566 18134 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:04:07.938438 17140 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.933748 18149 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.942937 18142 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
W20260709 13:04:07.942538 17066 leader_election.cc:336] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:07.943889 17066 leader_election.cc:336] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.943974 18142 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:07.947708 17141 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:07.950191 18131 raft_consensus.cc:2749] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.952868 18134 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.949786 18142 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:07.966044 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:07.967103 17101 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
W20260709 13:04:07.978672 17066 leader_election.cc:336] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.982968 18131 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:07.983920 18131 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:07.986629 18131 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:07.994282 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:04:07.997547 17140 leader_election.cc:336] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:07.999397 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:08.000453 17167 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 3.
I20260709 13:04:08.001968 17167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:08.003031 18143 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.003885 18143 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:04:08.005600 17141 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.007596 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.006757 18143 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.008844 18143 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:08.014688 17140 leader_election.cc:336] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.019052 17067 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:04:08.020650 18149 raft_consensus.cc:2749] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.029577 17067 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.030452 18142 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.030962 17067 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, 8267993b082f449891e464bc4dee65f7
I20260709 13:04:08.031488 18142 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.032814 18140 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:08.033272 18140 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.033697 18140 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.035148 18142 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.037732 18152 raft_consensus.cc:2749] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:08.052194 17140 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.053683 17141 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.058684 18140 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.061403 18097 raft_consensus.cc:2804] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:08.090509 18097 raft_consensus.cc:493] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.090977 18097 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:08.066887 17140 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.079619 18152 raft_consensus.cc:493] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.097947 18152 raft_consensus.cc:515] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.101960 18152 leader_election.cc:290] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:08.108057 17066 leader_election.cc:336] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.111792 18131 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.112502 18131 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.115121 18131 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:08.123528 17067 leader_election.cc:336] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.124152 17067 leader_election.cc:304] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.124936 18152 raft_consensus.cc:2749] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:08.136610 17141 leader_election.cc:336] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.137168 17141 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.138046 18131 raft_consensus.cc:2749] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:08.138407 17141 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.138929 17141 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.139710 18131 raft_consensus.cc:2749] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.140336 18140 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 election: Requested vote from peers 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:08.141026 17066 leader_election.cc:336] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.143333 17140 leader_election.cc:336] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.123610 18097 raft_consensus.cc:515] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.147006 18097 leader_election.cc:290] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:08.151381 18097 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.152319 18097 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.155885 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:08.156689 17101 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:04:08.160717 18131 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.161597 18131 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.163987 18131 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:08.160687 17140 leader_election.cc:336] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.173429 18097 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.176975 17141 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.178610 18143 raft_consensus.cc:2804] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:08.179057 18143 raft_consensus.cc:493] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.179471 18143 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.179275 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fcf0cdbc04350a9368ac4f9972f72" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:08.181684 17169 raft_consensus.cc:3060] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.182476 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16ada941539640f58a0f2ba47d6c1954" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:08.182986 18131 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.183151 17101 raft_consensus.cc:3060] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:08.186724 17141 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.187321 17141 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.188938 18153 raft_consensus.cc:2749] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.190572 17099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:08.191287 17099 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:04:08.193411 18131 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.202844 18131 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.197113 17141 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:08.199126 18143 raft_consensus.cc:515] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.210165 18156 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:08.210644 18156 raft_consensus.cc:493] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.211037 18156 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:08.211550 17140 leader_election.cc:336] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.212962 17140 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.216125 18143 leader_election.cc:290] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:08.217723 17168 raft_consensus.cc:1275] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:08.219636 18156 raft_consensus.cc:515] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.222074 18156 leader_election.cc:290] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:08.200759 17101 raft_consensus.cc:2468] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:04:08.200445 17169 raft_consensus.cc:2468] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 4.
W20260709 13:04:08.226854 17141 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.226696 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c79060c3294f4fbe036c1fe48ae9bf" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:08.227876 17141 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.228058 17100 raft_consensus.cc:3060] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.228703 18143 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.230713 18140 consensus_queue.cc:1048] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:08.234345 17140 leader_election.cc:336] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.235257 17067 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.236367 18140 raft_consensus.cc:2804] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:08.236909 18140 raft_consensus.cc:697] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:08.238580 18156 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:08.239281 18156 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.238211 18140 consensus_queue.cc:237] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.241480 18156 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.243321 17100 raft_consensus.cc:2468] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 3.
I20260709 13:04:08.245229 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2354d787dc43f58fd361f6b4832507" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:08.245929 17101 raft_consensus.cc:3060] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:08.246600 17066 consensus_peers.cc:597] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.249243 17140 leader_election.cc:336] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.249890 17140 leader_election.cc:336] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.252699 17140 leader_election.cc:336] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.270844 16846 catalog_manager.cc:5697] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 3 to 4, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 4 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.277009 17101 raft_consensus.cc:2468] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 4.
I20260709 13:04:08.285211 17141 leader_election.cc:304] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.286159 18156 raft_consensus.cc:2804] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:08.286681 18156 raft_consensus.cc:697] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:08.293457 18156 consensus_queue.cc:237] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.286577 17141 leader_election.cc:304] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.300017 18143 raft_consensus.cc:2804] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:08.300576 18143 raft_consensus.cc:697] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:08.301610 18143 consensus_queue.cc:237] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.305094 17141 leader_election.cc:304] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c
I20260709 13:04:08.309640 18097 raft_consensus.cc:2804] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:08.310276 18097 raft_consensus.cc:697] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 4 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:08.311475 18097 consensus_queue.cc:237] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
W20260709 13:04:08.317886 17141 leader_election.cc:336] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.318360 17141 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.319124 18131 raft_consensus.cc:2749] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:08.332641 17141 leader_election.cc:336] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.333174 17141 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.336048 16846 catalog_manager.cc:5697] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 2 to 3, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.338675 18156 raft_consensus.cc:2749] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.338529 16846 catalog_manager.cc:5697] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 2 to 3, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 3 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.363322 16846 catalog_manager.cc:5697] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 3 to 4, leader changed from 64734f14219348f59d1cf8700416565c (127.7.117.129) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 4 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.376948 18149 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.377746 18149 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.380304 18149 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:08.383613 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:04:08.387470 17067 leader_election.cc:336] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.388037 17067 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.388856 18149 raft_consensus.cc:2749] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.392623 18149 raft_consensus.cc:493] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.393414 18149 raft_consensus.cc:515] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.396075 18149 leader_election.cc:290] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:08.401808 17066 leader_election.cc:336] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.402595 17067 leader_election.cc:336] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.403138 18152 raft_consensus.cc:493] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.403880 18152 raft_consensus.cc:515] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.404429 17067 leader_election.cc:304] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.405509 18149 raft_consensus.cc:2749] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.406309 18152 leader_election.cc:290] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.408942 18152 raft_consensus.cc:493] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.409543 18152 raft_consensus.cc:515] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:04:08.414060 17067 leader_election.cc:336] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.414974 18152 leader_election.cc:290] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:08.419617 17066 leader_election.cc:336] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.420159 17066 leader_election.cc:304] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.423249 18149 raft_consensus.cc:2749] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:08.429533 17067 leader_election.cc:336] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:08.430071 17066 leader_election.cc:336] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.430578 17066 leader_election.cc:304] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.431376 18149 raft_consensus.cc:2749] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.502152 18097 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.502820 18097 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.504858 18097 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:08.507117 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
W20260709 13:04:08.511600 17141 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.512097 17141 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.512854 18097 raft_consensus.cc:2749] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.579367 18149 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.580159 18149 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.583295 18149 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:08.590198 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
W20260709 13:04:08.598942 17067 leader_election.cc:336] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:08.603441  7638 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 17048)
I20260709 13:04:08.605590 17067 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 8267993b082f449891e464bc4dee65f7, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.609364 18149 raft_consensus.cc:2749] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.625097 18156 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.625869 18156 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.630541 18156 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:08.636300 18156 raft_consensus.cc:493] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.637020 18156 raft_consensus.cc:515] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:08.639635 18156 leader_election.cc:290] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:08.644541 17141 leader_election.cc:336] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.646173 18152 raft_consensus.cc:493] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.646947 18152 raft_consensus.cc:515] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:04:08.649439 17140 leader_election.cc:336] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.650156 18152 leader_election.cc:290] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.653672 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:08.654325 17101 raft_consensus.cc:2468] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:04:08.659391 17141 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.660259 18131 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.660912 18153 raft_consensus.cc:2804] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.660363 18143 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.661546 18153 raft_consensus.cc:493] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.662005 18153 raft_consensus.cc:3060] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.661031 18131 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.667183 18131 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.662020 18143 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.669999 18143 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:08.673394 17140 leader_election.cc:336] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.659711 18149 raft_consensus.cc:493] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.675699 18149 raft_consensus.cc:515] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:04:08.677619 17141 leader_election.cc:336] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.685531 17141 leader_election.cc:304] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.688606 18142 raft_consensus.cc:2749] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.690526 17100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5" is_pre_election: true
I20260709 13:04:08.691263 17100 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 1.
I20260709 13:04:08.682483 18153 raft_consensus.cc:515] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.695506 18149 leader_election.cc:290] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.697470 18153 leader_election.cc:290] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:08.695400 17141 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: 
I20260709 13:04:08.698853 18142 raft_consensus.cc:2804] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.699273 18142 raft_consensus.cc:493] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.699680 18142 raft_consensus.cc:3060] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.701192 17066 leader_election.cc:336] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.703665 18152 raft_consensus.cc:493] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.704355 18152 raft_consensus.cc:515] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.707576 18152 leader_election.cc:290] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.708227 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd447a4120b4d378b710f3f18d0541f" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:08.708889 17101 raft_consensus.cc:3060] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.719833 17066 leader_election.cc:336] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.720840 17141 leader_election.cc:336] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.721370 17141 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:08.722328 17141 leader_election.cc:336] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:08.722407 17067 leader_election.cc:336] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.724766 18156 raft_consensus.cc:2749] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:08.727533 17066 consensus_peers.cc:597] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.728261 17066 leader_election.cc:336] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.722971 17067 leader_election.cc:304] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.729677 18149 raft_consensus.cc:2749] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.730033 17101 raft_consensus.cc:2468] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
I20260709 13:04:08.730921 18153 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.731551 18153 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:08.733170 18142 raft_consensus.cc:515] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.735520 18142 leader_election.cc:290] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:08.736397 17141 leader_election.cc:336] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.738853 17101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9bdaab91b264f3ea1e21283b3c988ac" candidate_uuid: "8267993b082f449891e464bc4dee65f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1cc950e241a640e0912a73ed9309b1e5"
I20260709 13:04:08.739506 17101 raft_consensus.cc:3060] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.743762 17141 leader_election.cc:304] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:08.746678 17140 leader_election.cc:336] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.748306 18153 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.749269 18156 raft_consensus.cc:2804] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.750139 18156 raft_consensus.cc:697] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
W20260709 13:04:08.750174 17067 leader_election.cc:336] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.750864 17067 leader_election.cc:304] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.751642 18149 raft_consensus.cc:2749] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.751358 18156 consensus_queue.cc:237] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:04:08.752885 17067 leader_election.cc:336] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.753388 17067 leader_election.cc:304] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.754139 18149 raft_consensus.cc:2749] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.760016 17101 raft_consensus.cc:2468] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8267993b082f449891e464bc4dee65f7 in term 2.
W20260709 13:04:08.769147 17140 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.773680 17140 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.770779 17141 leader_election.cc:336] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.781250 17141 leader_election.cc:304] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.782132 18156 raft_consensus.cc:2804] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.782735 18156 raft_consensus.cc:697] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Becoming Leader. State: Replica: 8267993b082f449891e464bc4dee65f7, State: Running, Role: LEADER
I20260709 13:04:08.781692 16846 catalog_manager.cc:5697] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:08.783809 18156 consensus_queue.cc:237] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:04:08.788612 17141 leader_election.cc:336] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.789156 17141 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.793115 18097 raft_consensus.cc:2749] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.811311 17168 raft_consensus.cc:1275] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:08.806798 16846 catalog_manager.cc:5697] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 reported cstate change: term changed from 1 to 2, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 8267993b082f449891e464bc4dee65f7 (127.7.117.132). New cstate: current_term: 2 leader_uuid: "8267993b082f449891e464bc4dee65f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:08.820577 18152 consensus_queue.cc:1048] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.822866 18152 raft_consensus.cc:493] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.823464 18152 raft_consensus.cc:515] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
W20260709 13:04:08.834044 17067 leader_election.cc:336] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:08.835345 17140 consensus_peers.cc:597] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.835744 18152 leader_election.cc:290] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:08.838848 17101 raft_consensus.cc:1275] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:08.839418 18152 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.840090 18152 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:08.844261 18152 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
W20260709 13:04:08.845175 17067 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.846282 17101 raft_consensus.cc:1275] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:08.846887 17169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7" is_pre_election: true
I20260709 13:04:08.847589 17169 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 1.
I20260709 13:04:08.847489 18097 consensus_queue.cc:1048] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.849845 17067 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:08.850894 17066 leader_election.cc:336] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.851431 17066 leader_election.cc:304] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.851446 18152 raft_consensus.cc:2804] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.852530 18152 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.852856 18149 raft_consensus.cc:2749] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.857421 18149 raft_consensus.cc:493] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.852960 18152 raft_consensus.cc:3060] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.858372 18149 raft_consensus.cc:515] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.860562 18149 leader_election.cc:290] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.864107 18097 consensus_queue.cc:1048] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:04:08.866859 17066 leader_election.cc:336] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:08.869814 18152 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:04:08.885893 17066 consensus_peers.cc:597] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:08.888103 17067 leader_election.cc:336] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.888612 17067 leader_election.cc:304] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.878150 18152 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Requested vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 8267993b082f449891e464bc4dee65f7 (127.7.117.132:38527)
I20260709 13:04:08.896507 18149 raft_consensus.cc:2749] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.896807 18168 raft_consensus.cc:493] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:08.897464 18168 raft_consensus.cc:515] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } }
I20260709 13:04:08.899544 18168 leader_election.cc:290] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:08.906239 17168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71c71675c2b420aaaad7cd24eef4084" candidate_uuid: "1cc950e241a640e0912a73ed9309b1e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8267993b082f449891e464bc4dee65f7"
I20260709 13:04:08.906976 17168 raft_consensus.cc:3060] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.908524 17066 leader_election.cc:336] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:08.902637 17067 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.916694 17168 raft_consensus.cc:2468] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc950e241a640e0912a73ed9309b1e5 in term 2.
W20260709 13:04:08.917927 17140 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:08.922065 17067 leader_election.cc:336] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:08.923033 17067 leader_election.cc:304] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.924091 18168 raft_consensus.cc:2749] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.926541 17067 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cc950e241a640e0912a73ed9309b1e5, 8267993b082f449891e464bc4dee65f7; no voters: ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:08.927359 18168 raft_consensus.cc:2804] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.927834 18168 raft_consensus.cc:697] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Becoming Leader. State: Replica: 1cc950e241a640e0912a73ed9309b1e5, State: Running, Role: LEADER
I20260709 13:04:08.928828 18168 consensus_queue.cc:237] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
W20260709 13:04:08.955370 17140 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:08.943594 17098 raft_consensus.cc:1275] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Refusing update from remote peer 8267993b082f449891e464bc4dee65f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:08.963307 18153 consensus_queue.cc:1048] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:08.974550 17140 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:08.984640 17066 consensus_peers.cc:597] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.004217 16846 catalog_manager.cc:5697] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 reported cstate change: term changed from 1 to 2, leader changed from ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130) to 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1cc950e241a640e0912a73ed9309b1e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:09.026124 17066 consensus_peers.cc:597] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.029340 17140 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.055054 17140 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.082899 17066 consensus_peers.cc:597] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.102626 17066 consensus_peers.cc:597] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.108321  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
W20260709 13:04:09.196374 17066 consensus_peers.cc:597] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.230551 17066 consensus_peers.cc:597] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.249550  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
W20260709 13:04:09.266292 17141 consensus_peers.cc:597] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.291700 17141 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.295724  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:09.298252  7638 tablet_replica.cc:333] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.299093  7638 raft_consensus.cc:2243] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.299624  7638 raft_consensus.cc:2272] T eb98a7c474ea41c3a428ce29909fc94f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.302615  7638 tablet_replica.cc:333] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
W20260709 13:04:09.303149 17141 consensus_peers.cc:597] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.303622  7638 raft_consensus.cc:2243] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.304114  7638 raft_consensus.cc:2272] T a12c1a9abfa94488a9aefb37cb9ecdf0 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.306847  7638 tablet_replica.cc:333] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.307794  7638 raft_consensus.cc:2243] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.308538  7638 raft_consensus.cc:2272] T 0eccff1f699a4e4387998ef0842e7d33 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.311369  7638 tablet_replica.cc:333] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.312213  7638 raft_consensus.cc:2243] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.312778  7638 raft_consensus.cc:2272] T e14794cbc74f4fcf8d3872e3463ad58d P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.315457  7638 tablet_replica.cc:333] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
W20260709 13:04:09.315814 17141 consensus_peers.cc:597] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.316334  7638 raft_consensus.cc:2243] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.316891  7638 raft_consensus.cc:2272] T edd447a4120b4d378b710f3f18d0541f P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.319198  7638 tablet_replica.cc:333] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.319900  7638 raft_consensus.cc:2243] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.320884  7638 raft_consensus.cc:2272] T 8fd44ab20c334b99964d344244e650ad P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.323375  7638 tablet_replica.cc:333] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.324010  7638 raft_consensus.cc:2243] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.324883  7638 raft_consensus.cc:2272] T 391757a5f74549289d0b597a84df6740 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.327378  7638 tablet_replica.cc:333] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.328037  7638 raft_consensus.cc:2243] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.328961  7638 raft_consensus.cc:2272] T 2ec250363e734059ae59f362df78c792 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.331321  7638 tablet_replica.cc:333] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.331959  7638 raft_consensus.cc:2243] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.332448  7638 raft_consensus.cc:2272] T a40bdee1b5c3486b94412af27867b738 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.334653  7638 tablet_replica.cc:333] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.335206  7638 raft_consensus.cc:2243] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.336057  7638 raft_consensus.cc:2272] T 4ed6b9a59b31479aaa96d38781de77d6 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.338104  7638 tablet_replica.cc:333] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.338961  7638 raft_consensus.cc:2243] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.340201  7638 raft_consensus.cc:2272] T 8f8e87277d714ded843585986eae389e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.342675  7638 tablet_replica.cc:333] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.343487  7638 raft_consensus.cc:2243] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.344039  7638 raft_consensus.cc:2272] T 61ae0ddb123d4d8bb5b071178e546931 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.347122  7638 tablet_replica.cc:333] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.347865  7638 raft_consensus.cc:2243] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.348356  7638 raft_consensus.cc:2272] T 0c8c7883a65740a5842d54e35490a6e1 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.350487  7638 tablet_replica.cc:333] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.351030  7638 raft_consensus.cc:2243] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.351495  7638 raft_consensus.cc:2272] T bafbaa58b13148c2b916fe7ee7e234e6 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.353628  7638 tablet_replica.cc:333] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.354185  7638 raft_consensus.cc:2243] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.354722  7638 raft_consensus.cc:2272] T ed56f28edb0c443ead4f92414e0a29fa P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.357271  7638 tablet_replica.cc:333] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.358095  7638 raft_consensus.cc:2243] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.358578  7638 raft_consensus.cc:2272] T 60c79060c3294f4fbe036c1fe48ae9bf P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.360773 17141 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.362085  7638 tablet_replica.cc:333] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.363221  7638 raft_consensus.cc:2243] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.364405  7638 raft_consensus.cc:2272] T d3c68d18d16941a9b3197081006477dc P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.366338 17141 consensus_peers.cc:597] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.367355  7638 tablet_replica.cc:333] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.367892 17168 raft_consensus.cc:1275] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Refusing update from remote peer 1cc950e241a640e0912a73ed9309b1e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.368295  7638 raft_consensus.cc:2243] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.369211 18149 consensus_queue.cc:1048] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:09.369568  7638 raft_consensus.cc:2272] T 4633fd2f819d45d893ebcdb54511d834 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.372941  7638 tablet_replica.cc:333] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.374015  7638 raft_consensus.cc:2243] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.375614  7638 raft_consensus.cc:2272] T 9ed0bc1914444d179b36b1253f15c65c P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.379524  7638 tablet_replica.cc:333] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.380496  7638 raft_consensus.cc:2243] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.381712  7638 raft_consensus.cc:2272] T 1ae68a5d259b476daae7a5665770a0cd P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.384193  7638 tablet_replica.cc:333] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.384850  7638 raft_consensus.cc:2243] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.386207  7638 raft_consensus.cc:2272] T 7d970e7113e5440181d65961915967ad P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.386510 17141 consensus_peers.cc:597] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.389549  7638 tablet_replica.cc:333] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.396869  7638 raft_consensus.cc:2243] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.397578  7638 raft_consensus.cc:2272] T 16ada941539640f58a0f2ba47d6c1954 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.401640 17140 consensus_peers.cc:597] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.402737 17067 consensus_peers.cc:597] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.404474  7638 tablet_replica.cc:333] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.405507  7638 raft_consensus.cc:2243] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.406504  7638 raft_consensus.cc:2272] T a9bdaab91b264f3ea1e21283b3c988ac P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.411504  7638 tablet_replica.cc:333] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.412942  7638 raft_consensus.cc:2243] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.414548  7638 raft_consensus.cc:2272] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.416769 17141 consensus_peers.cc:597] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.418213  7638 tablet_replica.cc:333] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.419173  7638 raft_consensus.cc:2243] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.419818  7638 raft_consensus.cc:2272] T 5b17e18d23d343e6894f11dc02b2eb75 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.423113  7638 tablet_replica.cc:333] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.423947  7638 raft_consensus.cc:2243] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.424482  7638 raft_consensus.cc:2272] T bd2354d787dc43f58fd361f6b4832507 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.427989  7638 tablet_replica.cc:333] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.428843  7638 raft_consensus.cc:2243] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.430126  7638 raft_consensus.cc:2272] T 162f7d6234e549cc968efac4ee8846ac P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.433096  7638 tablet_replica.cc:333] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.434010  7638 raft_consensus.cc:2243] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.434690  7638 raft_consensus.cc:2272] T d4035d55f3884082a4ba26e608bfdd6b P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.437942  7638 tablet_replica.cc:333] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
W20260709 13:04:09.438442 17141 consensus_peers.cc:597] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.438874  7638 raft_consensus.cc:2243] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:09.439613 17140 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.440128  7638 raft_consensus.cc:2272] T c71c71675c2b420aaaad7cd24eef4084 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.442855  7638 tablet_replica.cc:333] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.443449  7638 raft_consensus.cc:2243] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.443934  7638 raft_consensus.cc:2272] T 3ca37e8a20c2447983a81703b8706d51 P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.446475  7638 tablet_replica.cc:333] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.447050  7638 raft_consensus.cc:2243] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.447571  7638 raft_consensus.cc:2272] T 0395376e4b3d44a0a28a7bb199a21307 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.450095  7638 tablet_replica.cc:333] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.450845  7638 raft_consensus.cc:2243] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.451520  7638 raft_consensus.cc:2272] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.452886 17140 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.454562  7638 tablet_replica.cc:333] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.455269  7638 raft_consensus.cc:2243] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.456118  7638 raft_consensus.cc:2272] T bf62f622f0524a91bf77aab05b4c9d3f P 1cc950e241a640e0912a73ed9309b1e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.458324  7638 tablet_replica.cc:333] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.458890  7638 raft_consensus.cc:2243] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.459401  7638 raft_consensus.cc:2272] T 2db96a4ed139460aa021acdd63b9b529 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.463009  7638 tablet_replica.cc:333] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.463872  7638 raft_consensus.cc:2243] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.465081  7638 raft_consensus.cc:2272] T 3e944c9ade604302b16ffd9d285523a4 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.469964  7638 tablet_replica.cc:333] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
W20260709 13:04:09.470584 17141 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.471012  7638 raft_consensus.cc:2243] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.471814  7638 raft_consensus.cc:2272] T 355dd8ce521a412c8b28a5988e30bf7e P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.475124  7638 tablet_replica.cc:333] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.475947  7638 raft_consensus.cc:2243] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.476629  7638 raft_consensus.cc:2272] T 006cb6681bb949508a3ac253c5a64661 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.480141  7638 tablet_replica.cc:333] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.480995  7638 raft_consensus.cc:2243] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.481611  7638 raft_consensus.cc:2272] T e797287dc34d42e3838f8f5a3ba68985 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.484762  7638 tablet_replica.cc:333] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.485540  7638 raft_consensus.cc:2243] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.486121  7638 raft_consensus.cc:2272] T 117045952bb14176abd739933fa51259 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.489068  7638 tablet_replica.cc:333] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.489845  7638 raft_consensus.cc:2243] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.490422  7638 raft_consensus.cc:2272] T afb0f82169da45819893f31678f63f76 P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.491407 17140 consensus_peers.cc:597] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.493386  7638 tablet_replica.cc:333] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.494257  7638 raft_consensus.cc:2243] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.495496  7638 raft_consensus.cc:2272] T 855f8184b9834eeb8c9957f6a2df61f4 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.498869  7638 tablet_replica.cc:333] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
W20260709 13:04:09.498863 17141 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.499768  7638 raft_consensus.cc:2243] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.500973  7638 raft_consensus.cc:2272] T a97fcf0cdbc04350a9368ac4f9972f72 P 1cc950e241a640e0912a73ed9309b1e5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.503566  7638 tablet_replica.cc:333] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.504230  7638 raft_consensus.cc:2243] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:09.505344  7638 raft_consensus.cc:2272] T accaf859802f485d9a8e81e692ccdb50 P 1cc950e241a640e0912a73ed9309b1e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:09.507800  7638 tablet_replica.cc:333] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5: stopping tablet replica
I20260709 13:04:09.508242  7638 raft_consensus.cc:2243] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:09.508664  7638 raft_consensus.cc:2272] T f0c2977ecbd943b3bbffc28c746df99f P 1cc950e241a640e0912a73ed9309b1e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:09.525156 17141 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.545826 17141 consensus_peers.cc:597] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.556636 17141 consensus_peers.cc:597] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.588021 18156 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:09.588658 18156 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:09.591512 18156 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:09.595924 17140 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:09.596516 17141 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:09.596864 17141 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:09.597482 18156 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:09.605571 17141 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.626469 17141 consensus_peers.cc:597] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.640187 17141 consensus_peers.cc:597] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.640645 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.676200 18156 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:09.676826 18156 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:09.679989 18156 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:09.681753 17141 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.683219 17140 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:09.683771 17141 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:09.684123 17141 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:09.684708 18156 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:09.704012 17141 consensus_peers.cc:597] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.741048 17141 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.750537 17140 consensus_peers.cc:597] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.772112 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.790426 18156 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:09.791307 18156 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:09.794634 18156 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:09.800174 17140 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:09.803862 17141 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:09.804435 17141 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:09.805126 18143 raft_consensus.cc:2749] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.931283 18143 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:09.931818 18143 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:04:09.932782 17141 consensus_peers.cc:597] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.933717 18143 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:09.937335 17140 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:09.937778 17141 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:09.938236 17141 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:09.938957 18143 raft_consensus.cc:2749] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:10.008605 17141 consensus_peers.cc:597] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:10.017721 18143 raft_consensus.cc:493] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:10.018299 18143 raft_consensus.cc:515] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.020713 18143 leader_election.cc:290] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.024510 17140 leader_election.cc:336] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.024945 17141 leader_election.cc:336] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.025403 17141 leader_election.cc:304] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.026142 18143 raft_consensus.cc:2749] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:10.067328 17141 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:10.158955 17141 consensus_peers.cc:597] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:10.162299 18143 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:10.162840 18143 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.164928 18143 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.168604 17140 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.169072 17141 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.169507 17141 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.170101 18143 raft_consensus.cc:2749] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:10.192217 17141 consensus_peers.cc:597] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:10.206355 18143 raft_consensus.cc:493] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:10.207072 18143 raft_consensus.cc:515] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.209357 18143 leader_election.cc:290] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.212626 17141 leader_election.cc:336] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.213152 17140 leader_election.cc:336] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.213531 17140 leader_election.cc:304] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.214113 18143 raft_consensus.cc:2749] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.328419 18176 raft_consensus.cc:493] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:10.328895 18176 raft_consensus.cc:515] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.330811 18176 leader_election.cc:290] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.334199 17140 leader_election.cc:336] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.335208 17141 leader_election.cc:336] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.335542 17141 leader_election.cc:304] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.336091 18176 raft_consensus.cc:2749] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.367084 18176 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:10.367698 18176 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.370113 18176 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.377162 17140 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.377951 17141 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.378331 17141 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.378918 18156 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.431970 18156 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.432453 18156 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.434249 18156 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.437891 17141 leader_election.cc:336] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.438846 17141 leader_election.cc:336] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.439271 17141 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.439894 18156 raft_consensus.cc:2749] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.461845 18176 raft_consensus.cc:493] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.462476 18176 raft_consensus.cc:515] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.464745 18176 leader_election.cc:290] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.469309 17140 leader_election.cc:336] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.470602 17141 leader_election.cc:336] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.470938 17141 leader_election.cc:304] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:10.471527 18176 raft_consensus.cc:2749] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.509393 18176 raft_consensus.cc:493] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:10.509908 18176 raft_consensus.cc:515] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.511987 18176 leader_election.cc:290] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:10.514958 18156 raft_consensus.cc:493] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ece68ef4a10b4f60b2f20b1e5c0a1ab4)
I20260709 13:04:10.515553 18156 raft_consensus.cc:515] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:04:10.517454 17140 leader_election.cc:336] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.517724 18156 leader_election.cc:290] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.522541 17141 leader_election.cc:336] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.523754 17140 leader_election.cc:336] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.523787 17141 leader_election.cc:304] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:10.524928 17141 leader_election.cc:336] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.525453 18176 raft_consensus.cc:2749] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.525964 17141 leader_election.cc:304] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.526778 18176 raft_consensus.cc:2749] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.531454 18176 raft_consensus.cc:493] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.532001 18176 raft_consensus.cc:515] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.532804 18156 raft_consensus.cc:493] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:10.533370 18156 raft_consensus.cc:515] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:10.534226 18176 leader_election.cc:290] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:10.535179 18156 leader_election.cc:290] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:10.539870 17141 leader_election.cc:336] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.540274 17141 leader_election.cc:336] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.541750 17140 leader_election.cc:336] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.542021 17140 leader_election.cc:304] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:10.542248 17141 leader_election.cc:336] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.542709 18156 raft_consensus.cc:2749] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.542680 17141 leader_election.cc:304] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.544349 18176 raft_consensus.cc:2749] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:10.589403  7638 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 17122)
I20260709 13:04:10.605635 18156 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.606186 18156 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.607899 18156 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.612174 17141 leader_election.cc:336] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:10.612627 17140 leader_election.cc:336] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.613019 17140 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:10.613596 18156 raft_consensus.cc:2749] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.623795 18156 raft_consensus.cc:493] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.624630 18156 raft_consensus.cc:515] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.626956 18156 leader_election.cc:290] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.630466 17141 leader_election.cc:336] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.631150 17141 leader_election.cc:336] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.631524 17141 leader_election.cc:304] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.632216 18156 raft_consensus.cc:2749] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.733925 18156 raft_consensus.cc:493] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.734534 18156 raft_consensus.cc:515] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.736905 18156 leader_election.cc:290] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.741603 17141 leader_election.cc:336] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.742373 17141 leader_election.cc:336] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.742714 17141 leader_election.cc:304] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.743311 18156 raft_consensus.cc:2749] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.744140 18156 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.744606 18156 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.746752 18156 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.750213 17141 leader_election.cc:336] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:10.752161 17140 leader_election.cc:336] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.752549 17140 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:10.753119 18156 raft_consensus.cc:2749] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.778326 18156 raft_consensus.cc:493] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.778940 18156 raft_consensus.cc:515] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.781395 18156 leader_election.cc:290] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.787791 17141 leader_election.cc:336] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.791237 17141 leader_election.cc:336] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.791666 17141 leader_election.cc:304] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.792423 18156 raft_consensus.cc:2749] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.865885 18156 raft_consensus.cc:493] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.866346 18156 raft_consensus.cc:515] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.868033 18156 leader_election.cc:290] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.871917 17141 leader_election.cc:336] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:10.872406 17140 leader_election.cc:336] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:10.872859 17140 leader_election.cc:304] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:10.873608 18156 raft_consensus.cc:2749] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:10.904474 17140 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:10.908535 18156 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.909072 18156 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.910918 18156 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.914886 17141 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:10.915915 17141 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:10.916328 17141 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.916824 18156 raft_consensus.cc:493] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.917071 18176 raft_consensus.cc:2749] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.917414 18156 raft_consensus.cc:515] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:10.919422 18156 leader_election.cc:290] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:10.923178 17141 leader_election.cc:336] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:10.924118 17141 leader_election.cc:336] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.924461 17141 leader_election.cc:304] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:10.925000 18156 raft_consensus.cc:2749] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.941131 18176 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:10.941702 18176 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:10.943794 18176 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:10.947886 17140 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:10.949045 17141 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:10.949405 17141 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:10.949954 18176 raft_consensus.cc:2749] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:11.201670 17140 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.231477 17140 consensus_peers.cc:597] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.350597 17140 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:11.420032 17141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111) [suppressed 508 similar messages]
W20260709 13:04:11.513829 17140 consensus_peers.cc:597] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:11.563947 17140 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:11.564432 17140 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:11.589952  7638 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 17122)
W20260709 13:04:11.616171 17140 consensus_peers.cc:597] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.661442 18176 raft_consensus.cc:493] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:11.662096 18176 raft_consensus.cc:515] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:11.664384 18176 leader_election.cc:290] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:11.668646 17140 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:11.669116 17141 leader_election.cc:336] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:11.669586 17141 leader_election.cc:304] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:11.670370 18176 raft_consensus.cc:2749] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:11.723152 17141 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.808449 17140 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:11.819259 17141 consensus_peers.cc:597] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.832868 17141 consensus_peers.cc:597] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.834547 17141 consensus_peers.cc:597] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.837965 18176 raft_consensus.cc:493] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:11.838660 18176 raft_consensus.cc:515] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
W20260709 13:04:11.840512 17140 consensus_peers.cc:597] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.841068 18176 leader_election.cc:290] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:11.844602 17141 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:11.846938 17140 leader_election.cc:336] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:11.847380 17140 leader_election.cc:304] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:11.848085 18176 raft_consensus.cc:2749] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:11.878171 17141 consensus_peers.cc:597] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.899196 17141 consensus_peers.cc:597] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.904850 17141 consensus_peers.cc:597] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.919832 17141 consensus_peers.cc:597] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.958243 17141 consensus_peers.cc:597] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:11.999058 17141 consensus_peers.cc:597] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.077996 17141 consensus_peers.cc:597] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.099610 17140 consensus_peers.cc:597] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:12.106796 17141 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:12.132164 17141 consensus_peers.cc:597] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:12.135017 18156 raft_consensus.cc:493] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.135427 18156 raft_consensus.cc:515] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:12.136813 18156 leader_election.cc:290] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:12.141112 17141 leader_election.cc:336] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:12.142059 17141 leader_election.cc:336] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.142465 17141 leader_election.cc:304] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.143160 18156 raft_consensus.cc:2749] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:12.144680 17141 consensus_peers.cc:597] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.147033 17141 consensus_peers.cc:597] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.166903 17140 consensus_peers.cc:597] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.174945 18156 raft_consensus.cc:493] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.175346 18156 raft_consensus.cc:515] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:12.176721 18156 leader_election.cc:290] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:12.180667 17141 leader_election.cc:336] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:12.181103 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.181181 17140 leader_election.cc:336] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:12.181751 17140 leader_election.cc:304] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:12.182467 18176 raft_consensus.cc:2749] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.202893 18156 raft_consensus.cc:493] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:12.203419 18156 raft_consensus.cc:515] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:12.205504 18156 leader_election.cc:290] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:12.208735 17140 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:12.209919 17141 leader_election.cc:336] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.210276 17141 leader_election.cc:304] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.210863 18156 raft_consensus.cc:2749] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:12.262135 17141 consensus_peers.cc:597] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:12.276360 18156 raft_consensus.cc:493] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:12.276963 18156 raft_consensus.cc:515] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:12.279757 18156 leader_election.cc:290] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:12.282476 17140 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:12.286222 17141 consensus_peers.cc:597] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.286729 17141 consensus_peers.cc:597] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:12.287312 17141 leader_election.cc:336] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.287705 17141 leader_election.cc:304] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.288486 18156 raft_consensus.cc:2749] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:12.302906 17140 consensus_peers.cc:597] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 -> Peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Couldn't send request to peer 64734f14219348f59d1cf8700416565c. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.336012 18156 raft_consensus.cc:493] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:12.336479 18156 raft_consensus.cc:515] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:12.338122 18156 leader_election.cc:290] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
W20260709 13:04:12.340864 17141 consensus_peers.cc:597] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:12.341935 17140 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:12.344568 17141 leader_election.cc:336] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.344996 17141 leader_election.cc:304] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.345721 18156 raft_consensus.cc:2749] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:12.360296 17141 consensus_peers.cc:597] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 -> Peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Couldn't send request to peer 1cc950e241a640e0912a73ed9309b1e5. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:12.419577 18156 raft_consensus.cc:493] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.419992 18156 raft_consensus.cc:515] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:12.421640 18156 leader_election.cc:290] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
W20260709 13:04:12.425554 17141 leader_election.cc:336] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
W20260709 13:04:12.426049 17140 leader_election.cc:336] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:12.426612 17140 leader_election.cc:304] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:12.427498 18156 raft_consensus.cc:2749] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:12.473502 17141 consensus_peers.cc:597] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.480015  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 13:04:12.498149 18156 raft_consensus.cc:493] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.499022 18156 raft_consensus.cc:515] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:12.501899 18156 leader_election.cc:290] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
W20260709 13:04:12.506207 17141 leader_election.cc:336] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
W20260709 13:04:12.507079 17141 leader_election.cc:336] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.507491 17141 leader_election.cc:304] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.508221 18156 raft_consensus.cc:2749] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.561112  7638 tablet_server.cc:179] TabletServer@127.7.117.132:0 shutting down...
I20260709 13:04:12.601279  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:12.602434  7638 tablet_replica.cc:333] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.603210  7638 raft_consensus.cc:2243] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.603896  7638 raft_consensus.cc:2272] T 391757a5f74549289d0b597a84df6740 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.606545 18176 raft_consensus.cc:493] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.607445  7638 tablet_replica.cc:333] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.607082 18176 raft_consensus.cc:515] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:12.609436 18176 leader_election.cc:290] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:12.609769  7638 raft_consensus.cc:2243] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:12.610957 17141 consensus_peers.cc:597] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.611116  7638 raft_consensus.cc:2272] T 0eccff1f699a4e4387998ef0842e7d33 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:12.613616 17140 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
W20260709 13:04:12.614157 17141 leader_election.cc:336] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.614555 17141 leader_election.cc:304] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:12.614686  7638 tablet_replica.cc:333] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.615145 18176 raft_consensus.cc:2749] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:12.615505  7638 raft_consensus.cc:2243] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.616088  7638 raft_consensus.cc:2272] T 9c455336bc164af1af57c6148b97d553 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.618382  7638 tablet_replica.cc:333] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.619024  7638 raft_consensus.cc:2243] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.619531  7638 raft_consensus.cc:2272] T 8d2819a664db491aa04d5c6ec4c13708 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.621676  7638 tablet_replica.cc:333] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.622372  7638 raft_consensus.cc:2243] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.622843  7638 raft_consensus.cc:2272] T 8fd44ab20c334b99964d344244e650ad P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.622900 18176 raft_consensus.cc:493] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.623397 18176 raft_consensus.cc:515] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:12.625190 18176 leader_election.cc:290] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:12.625409  7638 tablet_replica.cc:333] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.626418  7638 raft_consensus.cc:2243] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.627575  7638 raft_consensus.cc:2272] T e5e1d0f0dc3240608259e975a8ea33ec P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:12.630772 17141 leader_election.cc:336] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.631256  7638 tablet_replica.cc:333] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.632274  7638 raft_consensus.cc:2243] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:12.632298 17141 leader_election.cc:336] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.632942  7638 raft_consensus.cc:2272] T a97fcf0cdbc04350a9368ac4f9972f72 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.633438 17141 leader_election.cc:304] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:12.634768 17141 consensus_peers.cc:597] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.635748 18176 raft_consensus.cc:2749] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.636422  7638 tablet_replica.cc:333] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.637406  7638 raft_consensus.cc:2243] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.637961  7638 raft_consensus.cc:2272] T 100511a7ba6744e08fb2dec95a371354 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.640653 18156 raft_consensus.cc:493] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64734f14219348f59d1cf8700416565c)
I20260709 13:04:12.641120  7638 tablet_replica.cc:333] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.641040 18156 raft_consensus.cc:515] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } } peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } }
I20260709 13:04:12.641969  7638 raft_consensus.cc:2243] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.642511  7638 raft_consensus.cc:2272] T 6a02330036da4716bc0da7d166bdea11 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.642748 18156 leader_election.cc:290] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 64734f14219348f59d1cf8700416565c (127.7.117.129:38761), ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149)
I20260709 13:04:12.645637  7638 tablet_replica.cc:333] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.646565  7638 raft_consensus.cc:2243] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:12.647179 17141 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.647203  7638 raft_consensus.cc:2272] T 4ed6b9a59b31479aaa96d38781de77d6 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:12.647696 17140 leader_election.cc:336] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:12.648057 17140 leader_election.cc:304] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 64734f14219348f59d1cf8700416565c, ece68ef4a10b4f60b2f20b1e5c0a1ab4
I20260709 13:04:12.648624 18156 raft_consensus.cc:2749] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:12.650416  7638 tablet_replica.cc:333] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.651384  7638 raft_consensus.cc:2243] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.652278  7638 raft_consensus.cc:2272] T c6ce161acdcd435a8f869ae50a6ea162 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.654918  7638 tablet_replica.cc:333] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.655526  7638 raft_consensus.cc:2243] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.656360  7638 raft_consensus.cc:2272] T 61ae0ddb123d4d8bb5b071178e546931 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.658708  7638 tablet_replica.cc:333] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.659368  7638 raft_consensus.cc:2243] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.660181  7638 raft_consensus.cc:2272] T 0c8c7883a65740a5842d54e35490a6e1 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.662460  7638 tablet_replica.cc:333] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.663049  7638 raft_consensus.cc:2243] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.663838  7638 raft_consensus.cc:2272] T bafbaa58b13148c2b916fe7ee7e234e6 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.665966  7638 tablet_replica.cc:333] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.666574  7638 raft_consensus.cc:2243] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.667435  7638 raft_consensus.cc:2272] T ed56f28edb0c443ead4f92414e0a29fa P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.670483 18156 raft_consensus.cc:493] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.670593  7638 tablet_replica.cc:333] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.670974 18156 raft_consensus.cc:515] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } } peers { permanent_uuid: "64734f14219348f59d1cf8700416565c" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 38761 } }
I20260709 13:04:12.671686  7638 raft_consensus.cc:2243] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.672688 18156 leader_election.cc:290] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115), 64734f14219348f59d1cf8700416565c (127.7.117.129:38761)
I20260709 13:04:12.672788  7638 raft_consensus.cc:2272] T e14794cbc74f4fcf8d3872e3463ad58d P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.676014  7638 tablet_replica.cc:333] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
W20260709 13:04:12.676795 17140 leader_election.cc:336] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64734f14219348f59d1cf8700416565c (127.7.117.129:38761): Network error: Client connection negotiation failed: client connection to 127.7.117.129:38761: connect: Connection refused (error 111)
I20260709 13:04:12.676916  7638 raft_consensus.cc:2243] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:12.677379 17141 leader_election.cc:336] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.677790 17141 leader_election.cc:304] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, 64734f14219348f59d1cf8700416565c
I20260709 13:04:12.678059  7638 pending_rounds.cc:70] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Trying to abort 1 pending ops.
I20260709 13:04:12.678406  7638 pending_rounds.cc:77] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305634810896429056 op_type: NO_OP noop_request { }
I20260709 13:04:12.678535 18156 raft_consensus.cc:2749] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.678834  7638 raft_consensus.cc:2889] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:12.679236  7638 raft_consensus.cc:2272] T edd447a4120b4d378b710f3f18d0541f P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.681376  7638 tablet_replica.cc:333] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.681941  7638 raft_consensus.cc:2243] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.682353  7638 raft_consensus.cc:2272] T cdaa7a451fbb426d868eef2a6abd8849 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.684412  7638 tablet_replica.cc:333] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.684921  7638 raft_consensus.cc:2243] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.685405  7638 raft_consensus.cc:2272] T a8756dd718c74c838e90911ac1fb65e5 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.687477  7638 tablet_replica.cc:333] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.687958  7638 raft_consensus.cc:2243] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.688364  7638 raft_consensus.cc:2272] T a9a2edef1b2a42bca289c34851377b55 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.690788  7638 tablet_replica.cc:333] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.691320  7638 raft_consensus.cc:2243] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.692088  7638 raft_consensus.cc:2272] T 60c79060c3294f4fbe036c1fe48ae9bf P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.694199  7638 tablet_replica.cc:333] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.694733  7638 raft_consensus.cc:2243] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.695118  7638 raft_consensus.cc:2272] T d3c68d18d16941a9b3197081006477dc P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.697088  7638 tablet_replica.cc:333] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.697721  7638 raft_consensus.cc:2243] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.698345  7638 raft_consensus.cc:2272] T 4633fd2f819d45d893ebcdb54511d834 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.701686  7638 tablet_replica.cc:333] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.702441  7638 raft_consensus.cc:2243] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.702919  7638 raft_consensus.cc:2272] T 7eae2b6552a54c4bbaff1296163ac17b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.704944  7638 tablet_replica.cc:333] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.705626  7638 raft_consensus.cc:2243] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.706096  7638 raft_consensus.cc:2272] T 162f7d6234e549cc968efac4ee8846ac P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.708257  7638 tablet_replica.cc:333] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.708880  7638 raft_consensus.cc:2243] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.709734  7638 raft_consensus.cc:2272] T d4035d55f3884082a4ba26e608bfdd6b P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.711375 18156 raft_consensus.cc:493] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1cc950e241a640e0912a73ed9309b1e5)
I20260709 13:04:12.711782 18156 raft_consensus.cc:515] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ece68ef4a10b4f60b2f20b1e5c0a1ab4" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42149 } } peers { permanent_uuid: "1cc950e241a640e0912a73ed9309b1e5" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 43115 } } peers { permanent_uuid: "8267993b082f449891e464bc4dee65f7" member_type: VOTER last_known_addr { host: "127.7.117.132" port: 38527 } }
I20260709 13:04:12.712399  7638 tablet_replica.cc:333] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.713434  7638 raft_consensus.cc:2243] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.714054  7638 raft_consensus.cc:2272] T 33c172d9d8e34d8b86a2d40dd97cfc0d P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.714241 18156 leader_election.cc:290] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149), 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115)
I20260709 13:04:12.717200  7638 tablet_replica.cc:333] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.718173  7638 raft_consensus.cc:2243] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:12.718536 17141 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1cc950e241a640e0912a73ed9309b1e5 (127.7.117.131:43115): Network error: Client connection negotiation failed: client connection to 127.7.117.131:43115: connect: Connection refused (error 111)
I20260709 13:04:12.719209  7638 pending_rounds.cc:70] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Trying to abort 1 pending ops.
I20260709 13:04:12.719532  7638 pending_rounds.cc:77] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305634811029573632 op_type: NO_OP noop_request { }
I20260709 13:04:12.719882  7638 raft_consensus.cc:2889] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:12.720191  7638 raft_consensus.cc:2272] T a9bdaab91b264f3ea1e21283b3c988ac P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:12.720511 17141 leader_election.cc:336] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111)
I20260709 13:04:12.720897 17141 leader_election.cc:304] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8267993b082f449891e464bc4dee65f7; no voters: 1cc950e241a640e0912a73ed9309b1e5, ece68ef4a10b4f60b2f20b1e5c0a1ab4
W20260709 13:04:12.721643 17141 consensus_peers.cc:597] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 -> Peer ece68ef4a10b4f60b2f20b1e5c0a1ab4 (127.7.117.130:42149): Couldn't send request to peer ece68ef4a10b4f60b2f20b1e5c0a1ab4. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:42149: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:12.721554 18176 raft_consensus.cc:2749] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.723059  7638 tablet_replica.cc:333] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.723758  7638 raft_consensus.cc:2243] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.724256  7638 raft_consensus.cc:2272] T 89b5b7c58e054ac6ac9e77e29ed86e5e P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.726754  7638 tablet_replica.cc:333] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.727500  7638 raft_consensus.cc:2243] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.728392  7638 raft_consensus.cc:2272] T 2db96a4ed139460aa021acdd63b9b529 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.730698  7638 tablet_replica.cc:333] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.731261  7638 raft_consensus.cc:2243] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.731729  7638 raft_consensus.cc:2272] T 3e944c9ade604302b16ffd9d285523a4 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.733881  7638 tablet_replica.cc:333] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.734452  7638 raft_consensus.cc:2243] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.734915  7638 raft_consensus.cc:2272] T c71c71675c2b420aaaad7cd24eef4084 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.737085  7638 tablet_replica.cc:333] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.737671  7638 raft_consensus.cc:2243] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.738467  7638 raft_consensus.cc:2272] T 3ca37e8a20c2447983a81703b8706d51 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.740365  7638 tablet_replica.cc:333] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.740876  7638 raft_consensus.cc:2243] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.741741  7638 raft_consensus.cc:2272] T 4da1ffde0abe4f029d91b01b8d08d2b7 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.743782  7638 tablet_replica.cc:333] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.744333  7638 raft_consensus.cc:2243] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.745139  7638 raft_consensus.cc:2272] T afe5ed1a667143e0869e8f06a2a27686 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.747083  7638 tablet_replica.cc:333] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.747639  7638 raft_consensus.cc:2243] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.748104  7638 raft_consensus.cc:2272] T 855f8184b9834eeb8c9957f6a2df61f4 P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.750118  7638 tablet_replica.cc:333] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.750659  7638 raft_consensus.cc:2243] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.751086  7638 raft_consensus.cc:2272] T 56ed34ea43214b00a76e89f8b09249de P 8267993b082f449891e464bc4dee65f7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.752951  7638 tablet_replica.cc:333] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.753545  7638 raft_consensus.cc:2243] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.753975  7638 raft_consensus.cc:2272] T bf62f622f0524a91bf77aab05b4c9d3f P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.755848  7638 tablet_replica.cc:333] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.756366  7638 raft_consensus.cc:2243] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.757133  7638 raft_consensus.cc:2272] T e797287dc34d42e3838f8f5a3ba68985 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.759136  7638 tablet_replica.cc:333] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.759672  7638 raft_consensus.cc:2243] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.760457  7638 raft_consensus.cc:2272] T afb0f82169da45819893f31678f63f76 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.762542  7638 tablet_replica.cc:333] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.763064  7638 raft_consensus.cc:2243] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.763481  7638 raft_consensus.cc:2272] T de112b6726e84fdca382fa7b9a903c49 P 8267993b082f449891e464bc4dee65f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.765498  7638 tablet_replica.cc:333] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.765978  7638 raft_consensus.cc:2243] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.766770  7638 raft_consensus.cc:2272] T 16ada941539640f58a0f2ba47d6c1954 P 8267993b082f449891e464bc4dee65f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.768546  7638 tablet_replica.cc:333] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.769021  7638 raft_consensus.cc:2243] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:12.769502  7638 raft_consensus.cc:2272] T 2abd5bac63004970887f5d75e88e9a0f P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.771585  7638 tablet_replica.cc:333] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7: stopping tablet replica
I20260709 13:04:12.772053  7638 raft_consensus.cc:2243] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.772828  7638 raft_consensus.cc:2272] T bd2354d787dc43f58fd361f6b4832507 P 8267993b082f449891e464bc4dee65f7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.840638  7638 tablet_server.cc:196] TabletServer@127.7.117.132:0 shutdown complete.
I20260709 13:04:12.905434  7638 master.cc:562] Master@127.7.117.190:42727 shutting down...
I20260709 13:04:12.950562  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:12.951128  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:12.951476  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1c7494c510344ab79b2f885aca8785d5: stopping tablet replica
I20260709 13:04:12.974509  7638 master.cc:584] Master@127.7.117.190:42727 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (35615 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:04:13.061415  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:35257
I20260709 13:04:13.062649  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:13.068189 18180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:13.068883 18181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:13.069762 18183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.070189  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:13.071224  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:13.071399  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:13.071520  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602253071509 us; error 0 us; skew 500 ppm
I20260709 13:04:13.071954  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:13.074249  7638 webserver.cc:533] Webserver started at http://127.7.117.190:45713/ using document root <none> and password file <none>
I20260709 13:04:13.074663  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:13.074810  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:13.075021  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:13.076162  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "91590c962bcb4bca8505ba0b2e81fd9e"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.080715  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:04:13.084038 18188 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.084757  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:13.085078  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "91590c962bcb4bca8505ba0b2e81fd9e"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.085413  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:13.097080  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:13.098305  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:13.135987  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:35257
I20260709 13:04:13.136093 18239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:35257 every 8 connection(s)
I20260709 13:04:13.139938 18240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.150472 18240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e: Bootstrap starting.
I20260709 13:04:13.155347 18240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:13.159792 18240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e: No bootstrap required, opened a new log
I20260709 13:04:13.162040 18240 raft_consensus.cc:359] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER }
I20260709 13:04:13.162410 18240 raft_consensus.cc:385] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:13.162678 18240 raft_consensus.cc:740] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91590c962bcb4bca8505ba0b2e81fd9e, State: Initialized, Role: FOLLOWER
I20260709 13:04:13.163363 18240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER }
I20260709 13:04:13.163812 18240 raft_consensus.cc:399] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:13.164039 18240 raft_consensus.cc:493] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:13.164305 18240 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:13.169003 18240 raft_consensus.cc:515] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER }
I20260709 13:04:13.169551 18240 leader_election.cc:304] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91590c962bcb4bca8505ba0b2e81fd9e; no voters: 
I20260709 13:04:13.170697 18240 leader_election.cc:290] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:13.171063 18243 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:13.172401 18243 raft_consensus.cc:697] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 1 LEADER]: Becoming Leader. State: Replica: 91590c962bcb4bca8505ba0b2e81fd9e, State: Running, Role: LEADER
I20260709 13:04:13.173097 18243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER }
I20260709 13:04:13.173728 18240 sys_catalog.cc:565] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:13.177645 18244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER } }
I20260709 13:04:13.178427 18244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:13.178190 18245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91590c962bcb4bca8505ba0b2e81fd9e. Latest consensus state: current_term: 1 leader_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91590c962bcb4bca8505ba0b2e81fd9e" member_type: VOTER } }
I20260709 13:04:13.180792 18245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:13.184010 18250 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:13.190145 18250 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:13.191449  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:13.199071 18250 catalog_manager.cc:1383] Generated new cluster ID: 4364fa463b2a4dcc8c439106134d2e71
I20260709 13:04:13.199353 18250 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:13.220105 18250 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:13.221441 18250 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:13.240798 18250 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e: Generated new TSK 0
I20260709 13:04:13.241447 18250 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:13.258481  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:13.264397 18261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:13.265812 18262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.267242  7638 server_base.cc:1061] running on GCE node
W20260709 13:04:13.267270 18264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.268442  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:13.268646  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:13.268847  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602253268812 us; error 0 us; skew 500 ppm
I20260709 13:04:13.269431  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:13.271661  7638 webserver.cc:533] Webserver started at http://127.7.117.129:39671/ using document root <none> and password file <none>
I20260709 13:04:13.272125  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:13.272320  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:13.272603  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:13.273778  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "921953afbbc9442ba1be12e26f1091c1"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.278225  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:04:13.281482 18269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.282186  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:13.282467  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "921953afbbc9442ba1be12e26f1091c1"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.282735  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:13.298789  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:13.300014  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:13.301604  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:13.303962  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:13.304184  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.304419  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:13.304618  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.344561  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:46283
I20260709 13:04:13.344656 18331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:46283 every 8 connection(s)
I20260709 13:04:13.349691  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:13.357144 18336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:13.357748 18337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.365701 18332 heartbeater.cc:344] Connected to a master server at 127.7.117.190:35257
W20260709 13:04:13.365737 18339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.366403  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:13.366434 18332 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:13.367466 18332 heartbeater.cc:507] Master 127.7.117.190:35257 requested a full tablet report, sending...
I20260709 13:04:13.367563  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:13.367928  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:13.368108  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602253368093 us; error 0 us; skew 500 ppm
I20260709 13:04:13.368872  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:13.369757 18205 ts_manager.cc:194] Registered new tserver with Master: 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283)
I20260709 13:04:13.371676  7638 webserver.cc:533] Webserver started at http://127.7.117.130:39499/ using document root <none> and password file <none>
I20260709 13:04:13.372334  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:13.372385 18205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59760
I20260709 13:04:13.372627  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:13.373009  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:13.374336  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "92d29f4c02f0400f8f67c124f88b8c10"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.378798  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:13.381981 18344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.382929  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:13.383256  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "92d29f4c02f0400f8f67c124f88b8c10"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.383545  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:13.405616  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:13.406793  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:13.408213  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:13.410429  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:13.410635  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.410897  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:13.411057  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.449465  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:41213
I20260709 13:04:13.449584 18406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:41213 every 8 connection(s)
I20260709 13:04:13.454238  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:13.461607 18410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:13.463636 18411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.464932  7638 server_base.cc:1061] running on GCE node
W20260709 13:04:13.466163 18413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:13.467083 18407 heartbeater.cc:344] Connected to a master server at 127.7.117.190:35257
I20260709 13:04:13.467125  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:13.467686  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:13.467724 18407 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:13.467926  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602253467912 us; error 0 us; skew 500 ppm
I20260709 13:04:13.468602  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:13.468631 18407 heartbeater.cc:507] Master 127.7.117.190:35257 requested a full tablet report, sending...
I20260709 13:04:13.470825 18205 ts_manager.cc:194] Registered new tserver with Master: 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:13.471210  7638 webserver.cc:533] Webserver started at http://127.7.117.131:36159/ using document root <none> and password file <none>
I20260709 13:04:13.471727  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:13.471927  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:13.472182  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:13.472441 18205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59768
I20260709 13:04:13.473371  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "bd62cdab312c454fb693d772d5305ec4"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.477881  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:04:13.480914 18418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.481748  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:13.482043  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "bd62cdab312c454fb693d772d5305ec4"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:13.482358  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:13.512560  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:13.513849  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:13.515316  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:13.517558  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:13.517786  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.518033  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:13.518208  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:13.559264  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:36683
I20260709 13:04:13.559388 18480 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:36683 every 8 connection(s)
I20260709 13:04:13.574021 18481 heartbeater.cc:344] Connected to a master server at 127.7.117.190:35257
I20260709 13:04:13.574419 18481 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:13.575207 18481 heartbeater.cc:507] Master 127.7.117.190:35257 requested a full tablet report, sending...
I20260709 13:04:13.577126 18205 ts_manager.cc:194] Registered new tserver with Master: bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:13.577435  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014749295s
I20260709 13:04:13.578610 18205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59776
I20260709 13:04:13.603010 18205 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59786:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:13.606055 18205 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:13.692953 18446 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0e6123188d48399753b8317fd4213e (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:13.694425 18446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0e6123188d48399753b8317fd4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.692968 18445 tablet_service.cc:1511] Processing CreateTablet for tablet d213c150e3ee4c2db44dcd864c8e15ab (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:13.695511 18445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d213c150e3ee4c2db44dcd864c8e15ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.698422 18444 tablet_service.cc:1511] Processing CreateTablet for tablet dcf45dde5c5a4bb8ab999adb039a8265 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:13.699891 18444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf45dde5c5a4bb8ab999adb039a8265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.702493 18443 tablet_service.cc:1511] Processing CreateTablet for tablet 6d115840e17448ae802e5efcbbb29590 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:13.704066 18443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d115840e17448ae802e5efcbbb29590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.714460 18442 tablet_service.cc:1511] Processing CreateTablet for tablet 4726c89fa9384720bf59d96ac827e73b (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:13.715974 18442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726c89fa9384720bf59d96ac827e73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.720952 18441 tablet_service.cc:1511] Processing CreateTablet for tablet 721b78f6d9594c61bf4e11677700dd2a (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:13.722452 18441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 721b78f6d9594c61bf4e11677700dd2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.727206 18297 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0e6123188d48399753b8317fd4213e (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:13.728688 18297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0e6123188d48399753b8317fd4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.732141 18440 tablet_service.cc:1511] Processing CreateTablet for tablet b62fc33c925046378913d49257ecfbe7 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:13.733572 18440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b62fc33c925046378913d49257ecfbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.734897 18296 tablet_service.cc:1511] Processing CreateTablet for tablet d213c150e3ee4c2db44dcd864c8e15ab (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:13.736313 18296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d213c150e3ee4c2db44dcd864c8e15ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.738348 18295 tablet_service.cc:1511] Processing CreateTablet for tablet dcf45dde5c5a4bb8ab999adb039a8265 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:13.739827 18295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf45dde5c5a4bb8ab999adb039a8265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.739959 18439 tablet_service.cc:1511] Processing CreateTablet for tablet b940b75365604200ad40c41249a0e28f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:13.741312 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b940b75365604200ad40c41249a0e28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.742585 18294 tablet_service.cc:1511] Processing CreateTablet for tablet 6d115840e17448ae802e5efcbbb29590 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:13.744184 18294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d115840e17448ae802e5efcbbb29590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.746639 18292 tablet_service.cc:1511] Processing CreateTablet for tablet 721b78f6d9594c61bf4e11677700dd2a (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:13.748272 18292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 721b78f6d9594c61bf4e11677700dd2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.750597 18291 tablet_service.cc:1511] Processing CreateTablet for tablet b62fc33c925046378913d49257ecfbe7 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:13.752267 18291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b62fc33c925046378913d49257ecfbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.754320 18293 tablet_service.cc:1511] Processing CreateTablet for tablet 4726c89fa9384720bf59d96ac827e73b (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:13.755722 18293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726c89fa9384720bf59d96ac827e73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.769425 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 3fc9957cf49e41f7b735627dd4bef580 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:13.771376 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc9957cf49e41f7b735627dd4bef580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.776608 18290 tablet_service.cc:1511] Processing CreateTablet for tablet b940b75365604200ad40c41249a0e28f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:13.780386 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b940b75365604200ad40c41249a0e28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.778949 18289 tablet_service.cc:1511] Processing CreateTablet for tablet 3fc9957cf49e41f7b735627dd4bef580 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:13.784116 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc9957cf49e41f7b735627dd4bef580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.788316 18445 tablet_service.cc:1511] Processing CreateTablet for tablet 2386e84784c94cae9251411d38311a1f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:13.789745 18445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386e84784c94cae9251411d38311a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.815778 18288 tablet_service.cc:1511] Processing CreateTablet for tablet 2386e84784c94cae9251411d38311a1f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:13.817276 18288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386e84784c94cae9251411d38311a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.821045 18372 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0e6123188d48399753b8317fd4213e (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:13.822505 18372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0e6123188d48399753b8317fd4213e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.826527 18371 tablet_service.cc:1511] Processing CreateTablet for tablet d213c150e3ee4c2db44dcd864c8e15ab (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:13.827976 18371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d213c150e3ee4c2db44dcd864c8e15ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.830915 18370 tablet_service.cc:1511] Processing CreateTablet for tablet dcf45dde5c5a4bb8ab999adb039a8265 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:13.832373 18370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf45dde5c5a4bb8ab999adb039a8265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.834542 18369 tablet_service.cc:1511] Processing CreateTablet for tablet 6d115840e17448ae802e5efcbbb29590 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:13.835958 18369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d115840e17448ae802e5efcbbb29590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.838541 18368 tablet_service.cc:1511] Processing CreateTablet for tablet 4726c89fa9384720bf59d96ac827e73b (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:13.839479 18366 tablet_service.cc:1511] Processing CreateTablet for tablet b62fc33c925046378913d49257ecfbe7 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:13.839996 18368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726c89fa9384720bf59d96ac827e73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.840859 18366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b62fc33c925046378913d49257ecfbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.842208 18367 tablet_service.cc:1511] Processing CreateTablet for tablet 721b78f6d9594c61bf4e11677700dd2a (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:13.843528 18367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 721b78f6d9594c61bf4e11677700dd2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.846387 18365 tablet_service.cc:1511] Processing CreateTablet for tablet b940b75365604200ad40c41249a0e28f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:13.847739 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b940b75365604200ad40c41249a0e28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.850826 18364 tablet_service.cc:1511] Processing CreateTablet for tablet 3fc9957cf49e41f7b735627dd4bef580 (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:13.852193 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc9957cf49e41f7b735627dd4bef580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.874708 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 2386e84784c94cae9251411d38311a1f (DEFAULT_TABLE table=test-workload [id=fd3a85750091492f86680c39e5ed2ba1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:13.876718 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386e84784c94cae9251411d38311a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:13.953997 18497 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:13.978456 18496 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:13.987108 18496 tablet_bootstrap.cc:654] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:13.988766 18498 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:13.987478 18497 tablet_bootstrap.cc:654] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:13.995023 18498 tablet_bootstrap.cc:654] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:13.997850 18496 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:13.997910 18497 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:13.998631 18496 ts_tablet_manager.cc:1403] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.020s	user 0.003s	sys 0.010s
I20260709 13:04:13.998631 18497 ts_tablet_manager.cc:1403] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.004s
I20260709 13:04:14.001008 18498 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.001528 18498 ts_tablet_manager.cc:1403] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.002s
I20260709 13:04:14.001880 18497 raft_consensus.cc:359] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.002128 18496 raft_consensus.cc:359] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.002676 18497 raft_consensus.cc:385] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.002805 18496 raft_consensus.cc:385] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.003028 18497 raft_consensus.cc:740] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.003182 18496 raft_consensus.cc:740] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.003839 18497 consensus_queue.cc:260] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.004030 18496 consensus_queue.cc:260] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.004330 18498 raft_consensus.cc:359] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.005113 18498 raft_consensus.cc:385] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.005435 18498 raft_consensus.cc:740] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.006462 18498 consensus_queue.cc:260] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.006808 18481 heartbeater.cc:499] Master 127.7.117.190:35257 was elected leader, sending a full tablet report...
I20260709 13:04:14.007525 18496 ts_tablet_manager.cc:1434] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.005s
I20260709 13:04:14.008962 18496 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.014894 18407 heartbeater.cc:499] Master 127.7.117.190:35257 was elected leader, sending a full tablet report...
I20260709 13:04:14.015921 18497 ts_tablet_manager.cc:1434] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 13:04:14.016717 18332 heartbeater.cc:499] Master 127.7.117.190:35257 was elected leader, sending a full tablet report...
I20260709 13:04:14.017064 18497 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.024289 18497 tablet_bootstrap.cc:654] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.025058 18496 tablet_bootstrap.cc:654] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.028484 18498 ts_tablet_manager.cc:1434] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 13:04:14.029856 18498 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.037073 18498 tablet_bootstrap.cc:654] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.039638 18496 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.039642 18497 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.040499 18497 ts_tablet_manager.cc:1403] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:04:14.040511 18496 ts_tablet_manager.cc:1403] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.008s
I20260709 13:04:14.043787 18497 raft_consensus.cc:359] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.044018 18496 raft_consensus.cc:359] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.044577 18497 raft_consensus.cc:385] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.044943 18497 raft_consensus.cc:740] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.044950 18496 raft_consensus.cc:385] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.045565 18496 raft_consensus.cc:740] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.045913 18497 consensus_queue.cc:260] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.046610 18496 consensus_queue.cc:260] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.048146 18497 ts_tablet_manager.cc:1434] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:04:14.049160 18496 ts_tablet_manager.cc:1434] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:04:14.049183 18497 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.050289 18496 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.056732 18497 tablet_bootstrap.cc:654] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.057147 18496 tablet_bootstrap.cc:654] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.063225 18497 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
W20260709 13:04:14.063658 18482 tablet.cc:2406] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:14.063858 18497 ts_tablet_manager.cc:1403] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.006s
I20260709 13:04:14.063341 18498 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.065606 18498 ts_tablet_manager.cc:1403] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.008s
I20260709 13:04:14.067224 18497 raft_consensus.cc:359] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.068763 18498 raft_consensus.cc:359] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.069185 18497 raft_consensus.cc:385] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.069581 18498 raft_consensus.cc:385] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.070336 18497 raft_consensus.cc:740] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.070719 18498 raft_consensus.cc:740] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.071408 18497 consensus_queue.cc:260] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.071592 18498 consensus_queue.cc:260] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.073963 18498 ts_tablet_manager.cc:1434] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:14.075162 18498 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.078516 18497 ts_tablet_manager.cc:1434] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:04:14.079689 18497 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.094317 18497 tablet_bootstrap.cc:654] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.094636 18498 tablet_bootstrap.cc:654] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.094470 18496 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.096941 18496 ts_tablet_manager.cc:1403] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.008s
I20260709 13:04:14.100418 18496 raft_consensus.cc:359] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.101318 18496 raft_consensus.cc:385] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.101730 18496 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.103003 18502 raft_consensus.cc:493] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.102728 18496 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.103559 18502 raft_consensus.cc:515] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.105511 18496 ts_tablet_manager.cc:1434] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:14.106802 18496 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.113521 18502 leader_election.cc:290] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.117952 18498 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.118877 18497 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.120504 18496 tablet_bootstrap.cc:654] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.121055 18497 ts_tablet_manager.cc:1403] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.008s
I20260709 13:04:14.121061 18498 ts_tablet_manager.cc:1403] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.011s
I20260709 13:04:14.124169 18498 raft_consensus.cc:359] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.124401 18497 raft_consensus.cc:359] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.124892 18498 raft_consensus.cc:385] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.125118 18497 raft_consensus.cc:385] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.125527 18497 raft_consensus.cc:740] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.126334 18497 consensus_queue.cc:260] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.125222 18498 raft_consensus.cc:740] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.128783 18497 ts_tablet_manager.cc:1434] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:04:14.128716 18498 consensus_queue.cc:260] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.129891 18497 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.131675 18496 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.132318 18496 ts_tablet_manager.cc:1403] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.000s
I20260709 13:04:14.135557 18496 raft_consensus.cc:359] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.136421 18496 raft_consensus.cc:385] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.136662 18497 tablet_bootstrap.cc:654] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.136844 18496 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.137694 18498 ts_tablet_manager.cc:1434] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:04:14.138089 18496 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.140831 18496 ts_tablet_manager.cc:1434] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:04:14.142095 18496 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.144671 18498 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.148671 18503 raft_consensus.cc:493] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.149197 18503 raft_consensus.cc:515] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.151257 18496 tablet_bootstrap.cc:654] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.151928 18503 leader_election.cc:290] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.152801 18498 tablet_bootstrap.cc:654] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.172675 18502 raft_consensus.cc:493] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.173524 18502 raft_consensus.cc:515] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.176677 18502 leader_election.cc:290] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.177439 18502 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.178087 18502 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.180800 18502 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
W20260709 13:04:14.189349 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:14.196579 18504 raft_consensus.cc:493] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.197427 18504 raft_consensus.cc:515] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.199389 18454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
W20260709 13:04:14.201434 18271 leader_election.cc:343] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.201839 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.203620 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
W20260709 13:04:14.207612 18271 leader_election.cc:343] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:14.208325 18271 leader_election.cc:343] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.209797 18504 leader_election.cc:290] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.214558 18496 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.215461 18496 ts_tablet_manager.cc:1403] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.074s	user 0.040s	sys 0.023s
I20260709 13:04:14.220592 18496 raft_consensus.cc:359] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.221765 18496 raft_consensus.cc:385] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.222263 18496 raft_consensus.cc:740] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.228523 18497 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.229393 18497 ts_tablet_manager.cc:1403] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.005s
I20260709 13:04:14.237870 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.238639 18382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
W20260709 13:04:14.239893 18421 leader_election.cc:343] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.240288 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.241264 18504 raft_consensus.cc:493] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.241109 18497 raft_consensus.cc:359] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.241817 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.241859 18504 raft_consensus.cc:515] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
W20260709 13:04:14.243588 18272 leader_election.cc:343] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.244119 18272 leader_election.cc:304] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1; no voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.245455 18502 raft_consensus.cc:2749] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.245652 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
W20260709 13:04:14.245849 18272 leader_election.cc:343] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.244626 18497 raft_consensus.cc:385] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.246248 18498 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.242679 18496 consensus_queue.cc:260] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.246343 18272 leader_election.cc:304] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1; no voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.247062 18498 ts_tablet_manager.cc:1403] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.103s	user 0.016s	sys 0.023s
I20260709 13:04:14.246344 18380 raft_consensus.cc:2468] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 0.
I20260709 13:04:14.248183 18502 raft_consensus.cc:2749] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.248719 18496 ts_tablet_manager.cc:1434] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:04:14.244161 18504 leader_election.cc:290] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
W20260709 13:04:14.249425 18272 leader_election.cc:343] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.249877 18496 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.249922 18272 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1; no voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.250944 18502 raft_consensus.cc:2749] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.250651 18498 raft_consensus.cc:359] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.252230 18498 raft_consensus.cc:385] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.253095 18498 raft_consensus.cc:740] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.253914 18498 consensus_queue.cc:260] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.255275 18421 leader_election.cc:304] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:14.251632 18497 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.257190 18503 raft_consensus.cc:2804] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.257654 18503 raft_consensus.cc:493] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.258028 18503 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.257807 18497 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.264314 18497 ts_tablet_manager.cc:1434] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:04:14.264657 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.265362 18307 raft_consensus.cc:2468] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.267082 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.268371 18503 raft_consensus.cc:515] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.268759 18497 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.268666 18347 leader_election.cc:304] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
W20260709 13:04:14.269832 18347 leader_election.cc:343] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.270707 18504 raft_consensus.cc:2804] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.271157 18504 raft_consensus.cc:493] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.271536 18504 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.275554 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.274432 18503 leader_election.cc:290] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.276542 18380 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.276791 18497 tablet_bootstrap.cc:654] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.277275 18496 tablet_bootstrap.cc:654] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.275566 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
W20260709 13:04:14.282943 18421 leader_election.cc:343] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.284621 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.286010 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.286804 18456 raft_consensus.cc:2468] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.288622 18496 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
W20260709 13:04:14.288729 18346 leader_election.cc:343] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.289358 18496 ts_tablet_manager.cc:1403] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.003s
I20260709 13:04:14.289942 18346 leader_election.cc:304] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:14.292078 18526 raft_consensus.cc:2804] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.292001 18496 raft_consensus.cc:359] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.292649 18526 raft_consensus.cc:493] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.292929 18496 raft_consensus.cc:385] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.293148 18526 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.293330 18496 raft_consensus.cc:740] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.294334 18496 consensus_queue.cc:260] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.296043 18504 raft_consensus.cc:515] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.298353 18497 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.298892 18498 ts_tablet_manager.cc:1434] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:04:14.298995 18497 ts_tablet_manager.cc:1403] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.011s
I20260709 13:04:14.301275 18526 raft_consensus.cc:515] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.302647 18496 ts_tablet_manager.cc:1434] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:14.302485 18497 raft_consensus.cc:359] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.303333 18497 raft_consensus.cc:385] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.303653 18497 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.303802 18496 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.304716 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.306043 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.306730 18456 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.304755 18526 leader_election.cc:290] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
W20260709 13:04:14.306164 18347 leader_election.cc:343] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.308728 18498 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.313199 18496 tablet_bootstrap.cc:654] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.315447 18456 raft_consensus.cc:2468] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.316803 18346 leader_election.cc:304] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:14.318066 18526 raft_consensus.cc:2804] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.304479 18497 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.319445 18380 raft_consensus.cc:2468] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 1.
I20260709 13:04:14.320612 18497 ts_tablet_manager.cc:1434] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:04:14.320787 18421 leader_election.cc:304] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:14.321674 18503 raft_consensus.cc:2804] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.321794 18497 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.323983 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.324854 18307 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.325224 18504 leader_election.cc:290] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.327383 18526 raft_consensus.cc:697] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:14.328405 18526 consensus_queue.cc:237] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.329687 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
W20260709 13:04:14.331274 18346 leader_election.cc:343] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:14.332269 18498 tablet_bootstrap.cc:654] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.336483 18307 raft_consensus.cc:2468] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.338275 18496 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.338290 18347 leader_election.cc:304] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.339035 18496 ts_tablet_manager.cc:1403] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.004s
I20260709 13:04:14.339550 18528 raft_consensus.cc:2804] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.340154 18528 raft_consensus.cc:697] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:14.341090 18528 consensus_queue.cc:237] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.341807 18496 raft_consensus.cc:359] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.342149 18503 raft_consensus.cc:697] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:14.343766 18503 consensus_queue.cc:237] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.345465 18497 tablet_bootstrap.cc:654] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.345652 18498 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.346359 18498 ts_tablet_manager.cc:1403] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.004s
I20260709 13:04:14.347173 18496 raft_consensus.cc:385] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.347496 18496 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.348390 18496 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.349792 18498 raft_consensus.cc:359] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.350590 18498 raft_consensus.cc:385] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.351111 18496 ts_tablet_manager.cc:1434] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:04:14.355216 18496 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.354490 18498 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.356215 18498 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.358309 18497 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.358549 18498 ts_tablet_manager.cc:1434] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:04:14.358912 18497 ts_tablet_manager.cc:1403] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.008s
I20260709 13:04:14.359864 18498 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.362375 18497 raft_consensus.cc:359] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.363438 18497 raft_consensus.cc:385] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.362380 18204 catalog_manager.cc:5697] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.363819 18497 raft_consensus.cc:740] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.364797 18497 consensus_queue.cc:260] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.362514 18496 tablet_bootstrap.cc:654] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.368646 18498 tablet_bootstrap.cc:654] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.372779 18497 ts_tablet_manager.cc:1434] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:04:14.374075 18497 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.378836 18496 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.379522 18496 ts_tablet_manager.cc:1403] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 13:04:14.383100 18497 tablet_bootstrap.cc:654] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.388991 18498 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.389629 18498 ts_tablet_manager.cc:1403] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.012s
I20260709 13:04:14.392318 18496 raft_consensus.cc:359] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.393059 18496 raft_consensus.cc:385] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.393350 18496 raft_consensus.cc:740] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.393886 18498 raft_consensus.cc:359] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.394170 18496 consensus_queue.cc:260] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.394695 18498 raft_consensus.cc:385] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.395159 18498 raft_consensus.cc:740] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.396050 18498 consensus_queue.cc:260] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.396505 18496 ts_tablet_manager.cc:1434] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:04:14.397637 18496 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.398696 18498 ts_tablet_manager.cc:1434] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:04:14.400708 18498 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.401372 18497 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.401876 18497 ts_tablet_manager.cc:1403] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.008s
I20260709 13:04:14.403082 18496 tablet_bootstrap.cc:654] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.408257 18498 tablet_bootstrap.cc:654] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.412732 18497 raft_consensus.cc:359] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.413633 18497 raft_consensus.cc:385] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.413940 18497 raft_consensus.cc:740] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.415014 18498 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.414773 18497 consensus_queue.cc:260] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.415612 18498 ts_tablet_manager.cc:1403] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.007s
I20260709 13:04:14.417448 18497 ts_tablet_manager.cc:1434] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:04:14.418545 18497 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.418932 18498 raft_consensus.cc:359] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.419711 18498 raft_consensus.cc:385] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.420004 18498 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.420802 18498 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.423333 18498 ts_tablet_manager.cc:1434] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:04:14.424391 18498 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.424553 18497 tablet_bootstrap.cc:654] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.430218 18496 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.430691 18496 ts_tablet_manager.cc:1403] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.011s
I20260709 13:04:14.431345 18497 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.431864 18497 ts_tablet_manager.cc:1403] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.014s	user 0.001s	sys 0.010s
I20260709 13:04:14.432013 18498 tablet_bootstrap.cc:654] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.433377 18496 raft_consensus.cc:359] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.434075 18496 raft_consensus.cc:385] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.434336 18496 raft_consensus.cc:740] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.434868 18497 raft_consensus.cc:359] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.435146 18496 consensus_queue.cc:260] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.435978 18497 raft_consensus.cc:385] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.436424 18497 raft_consensus.cc:740] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.432577 18202 catalog_manager.cc:5697] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.437660 18496 ts_tablet_manager.cc:1434] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:04:14.437821 18502 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.438663 18496 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: Bootstrap starting.
I20260709 13:04:14.436461 18203 catalog_manager.cc:5697] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.438587 18502 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.442863 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.443815 18380 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.444872 18496 tablet_bootstrap.cc:654] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.445921 18272 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.447252 18520 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.447788 18520 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.448231 18520 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.455614 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.456025 18496 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: No bootstrap required, opened a new log
I20260709 13:04:14.456413 18456 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.456565 18496 ts_tablet_manager.cc:1403] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:04:14.458015 18502 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.459568 18496 raft_consensus.cc:359] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.460253 18496 raft_consensus.cc:385] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.460506 18496 raft_consensus.cc:740] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.461305 18496 consensus_queue.cc:260] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.463538 18496 ts_tablet_manager.cc:1434] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:14.466012 18520 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.467870 18498 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.468400 18498 ts_tablet_manager.cc:1403] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.044s	user 0.027s	sys 0.011s
I20260709 13:04:14.469120 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.469786 18380 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.471364 18498 raft_consensus.cc:359] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.472139 18498 raft_consensus.cc:385] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.472189 18502 raft_consensus.cc:493] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.472544 18498 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.473366 18498 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.472771 18502 raft_consensus.cc:515] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.478772 18498 ts_tablet_manager.cc:1434] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.004s
I20260709 13:04:14.479803 18498 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.480680 18380 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 1.
I20260709 13:04:14.481537 18497 consensus_queue.cc:260] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.484740 18502 leader_election.cc:290] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.485337 18502 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.485968 18498 tablet_bootstrap.cc:654] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.485918 18502 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.486855 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.487938 18456 raft_consensus.cc:2393] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921953afbbc9442ba1be12e26f1091c1 in current term 1: Already voted for candidate 92d29f4c02f0400f8f67c124f88b8c10 in this term.
I20260709 13:04:14.488909 18502 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.490240 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.491128 18456 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.492750 18271 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.495076 18528 raft_consensus.cc:493] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.495781 18528 raft_consensus.cc:515] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.499724 18502 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.500907 18502 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.501437 18502 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.502331 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.503265 18307 raft_consensus.cc:2468] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.504940 18347 leader_election.cc:304] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.506450 18526 raft_consensus.cc:2804] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.506880 18526 raft_consensus.cc:493] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.507328 18526 raft_consensus.cc:3060] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.511269 18503 raft_consensus.cc:493] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.511952 18503 raft_consensus.cc:515] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.514683 18503 leader_election.cc:290] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.517690 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.518865 18307 raft_consensus.cc:2393] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd62cdab312c454fb693d772d5305ec4 in current term 1: Already voted for candidate 92d29f4c02f0400f8f67c124f88b8c10 in this term.
I20260709 13:04:14.520772 18528 leader_election.cc:290] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.507261 18497 ts_tablet_manager.cc:1434] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.000s
I20260709 13:04:14.522663 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.523577 18456 raft_consensus.cc:2468] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.529978 18520 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Requested vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.520510 18526 raft_consensus.cc:515] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.531432 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.532303 18456 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.533573 18526 leader_election.cc:290] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.534888 18497 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:14.535657 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.536491 18455 raft_consensus.cc:3060] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.545428 18497 tablet_bootstrap.cc:654] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.539031 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.552801 18421 leader_election.cc:304] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:14.549898 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "721b78f6d9594c61bf4e11677700dd2a" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.554291 18307 raft_consensus.cc:3060] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.555114 18503 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.555573 18502 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.566354 18455 raft_consensus.cc:2468] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.568203 18497 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: No bootstrap required, opened a new log
I20260709 13:04:14.569413 18272 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.569689 18497 ts_tablet_manager.cc:1403] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.005s
I20260709 13:04:14.570283 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.570915 18520 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.571300 18382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.571460 18520 raft_consensus.cc:697] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Becoming Leader. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Running, Role: LEADER
I20260709 13:04:14.572048 18497 raft_consensus.cc:359] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.572346 18520 consensus_queue.cc:237] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.572729 18497 raft_consensus.cc:385] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.572649 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.573367 18497 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.573725 18346 leader_election.cc:304] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.568205 18498 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.574571 18498 ts_tablet_manager.cc:1403] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.095s	user 0.040s	sys 0.036s
I20260709 13:04:14.574156 18497 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.575014 18526 raft_consensus.cc:2804] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.575519 18526 raft_consensus.cc:697] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:14.576295 18497 ts_tablet_manager.cc:1434] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:14.576983 18503 raft_consensus.cc:2749] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.570956 18381 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.577380 18498 raft_consensus.cc:359] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.577755 18272 leader_election.cc:304] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1; no voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.578342 18498 raft_consensus.cc:385] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.576333 18526 consensus_queue.cc:237] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.578706 18498 raft_consensus.cc:740] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.579264 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.579371 18502 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Requested vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.579938 18455 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.579465 18498 consensus_queue.cc:260] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.579283 18546 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.581946 18498 ts_tablet_manager.cc:1434] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:04:14.582937 18498 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: Bootstrap starting.
I20260709 13:04:14.562716 18307 raft_consensus.cc:2468] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.589318 18498 tablet_bootstrap.cc:654] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:14.589412 18456 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 1.
I20260709 13:04:14.590310 18502 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.591902 18502 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.592279 18455 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 1.
I20260709 13:04:14.593964 18271 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:14.595563 18547 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.596378 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.596800 18502 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.597074 18381 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.598449 18272 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.602424 18502 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.602824 18502 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.603188 18502 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.603760 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.604555 18498 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: No bootstrap required, opened a new log
I20260709 13:04:14.605448 18498 ts_tablet_manager.cc:1403] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.002s
I20260709 13:04:14.605414 18455 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 0.
I20260709 13:04:14.608871 18498 raft_consensus.cc:359] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.609602 18498 raft_consensus.cc:385] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:14.609865 18498 raft_consensus.cc:740] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Initialized, Role: FOLLOWER
I20260709 13:04:14.610553 18498 consensus_queue.cc:260] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.612154 18553 raft_consensus.cc:493] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.612609 18547 raft_consensus.cc:697] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Becoming Leader. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Running, Role: LEADER
I20260709 13:04:14.613129 18498 ts_tablet_manager.cc:1434] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:14.613749 18547 consensus_queue.cc:237] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.614560 18546 raft_consensus.cc:2749] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.616257 18202 catalog_manager.cc:5697] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "921953afbbc9442ba1be12e26f1091c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.617229 18553 raft_consensus.cc:515] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:14.620146 18502 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.620481 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.621707 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.622373 18502 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Requested vote from peers 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.622498 18381 raft_consensus.cc:2393] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921953afbbc9442ba1be12e26f1091c1 in current term 1: Already voted for candidate bd62cdab312c454fb693d772d5305ec4 in this term.
I20260709 13:04:14.624260 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.624814 18381 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.627091 18271 leader_election.cc:304] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1; no voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.628165 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "921953afbbc9442ba1be12e26f1091c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.628611 18455 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.629817 18201 catalog_manager.cc:5697] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.631027 18547 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.632979 18381 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 1.
I20260709 13:04:14.634143 18272 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.636153 18455 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921953afbbc9442ba1be12e26f1091c1 in term 1.
I20260709 13:04:14.637598 18546 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.638160 18546 raft_consensus.cc:697] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Becoming Leader. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Running, Role: LEADER
I20260709 13:04:14.639030 18546 consensus_queue.cc:237] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.639819 18202 catalog_manager.cc:5697] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "921953afbbc9442ba1be12e26f1091c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.642725 18553 leader_election.cc:290] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.645037 18547 raft_consensus.cc:2749] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.652035 18503 raft_consensus.cc:493] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.652560 18503 raft_consensus.cc:515] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.654346 18503 leader_election.cc:290] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.654364 18201 catalog_manager.cc:5697] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "921953afbbc9442ba1be12e26f1091c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.655186 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b940b75365604200ad40c41249a0e28f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.655455 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b940b75365604200ad40c41249a0e28f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.655812 18307 raft_consensus.cc:2468] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 0.
I20260709 13:04:14.656134 18381 raft_consensus.cc:2468] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 0.
I20260709 13:04:14.656838 18421 leader_election.cc:304] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.657676 18503 raft_consensus.cc:2804] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.657994 18503 raft_consensus.cc:493] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.658272 18503 raft_consensus.cc:3060] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.664079 18503 raft_consensus.cc:515] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.665657 18503 leader_election.cc:290] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.666466 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b940b75365604200ad40c41249a0e28f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.666739 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b940b75365604200ad40c41249a0e28f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.667014 18307 raft_consensus.cc:3060] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.667374 18381 raft_consensus.cc:3060] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.671779 18307 raft_consensus.cc:2468] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 1.
I20260709 13:04:14.672089 18381 raft_consensus.cc:2468] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 1.
I20260709 13:04:14.672928 18421 leader_election.cc:304] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.673806 18503 raft_consensus.cc:2804] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.674278 18503 raft_consensus.cc:697] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:14.675086 18503 consensus_queue.cc:237] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.682026 18201 catalog_manager.cc:5697] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.745958 18526 consensus_queue.cc:1048] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.757087 18528 consensus_queue.cc:1048] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.780112 18528 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.780550 18528 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.780993 18526 raft_consensus.cc:493] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.781594 18503 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.781613 18526 raft_consensus.cc:515] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.782303 18503 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.782698 18528 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.783732 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.784926 18307 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.786017 18503 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.786630 18306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.787025 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:14.787369 18306 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 0.
I20260709 13:04:14.787664 18526 leader_election.cc:290] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.787744 18381 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 0.
I20260709 13:04:14.788846 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.789492 18455 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.789662 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1" is_pre_election: true
I20260709 13:04:14.789798 18347 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.790323 18307 raft_consensus.cc:2468] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.790984 18528 raft_consensus.cc:2804] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.790870 18421 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.791463 18528 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.791481 18347 leader_election.cc:304] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:14.791828 18503 raft_consensus.cc:2804] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.791975 18528 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.792511 18526 raft_consensus.cc:2804] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:14.792586 18503 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.793061 18526 raft_consensus.cc:493] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.793220 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:14.793514 18526 raft_consensus.cc:3060] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.793218 18503 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.794082 18455 raft_consensus.cc:2468] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 0.
I20260709 13:04:14.797545 18504 consensus_queue.cc:1048] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.799714 18528 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.801061 18503 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.802930 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.803309 18528 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.803601 18307 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.803877 18526 raft_consensus.cc:515] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.805709 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.807049 18455 raft_consensus.cc:2393] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 92d29f4c02f0400f8f67c124f88b8c10 in current term 1: Already voted for candidate bd62cdab312c454fb693d772d5305ec4 in this term.
I20260709 13:04:14.811884 18307 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.813514 18503 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:14.814299 18528 consensus_queue.cc:1048] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.815810 18381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:14.816612 18381 raft_consensus.cc:2393] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd62cdab312c454fb693d772d5305ec4 in current term 1: Already voted for candidate 92d29f4c02f0400f8f67c124f88b8c10 in this term.
I20260709 13:04:14.818302 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.819151 18307 raft_consensus.cc:2393] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd62cdab312c454fb693d772d5305ec4 in current term 1: Already voted for candidate 92d29f4c02f0400f8f67c124f88b8c10 in this term.
I20260709 13:04:14.820215 18421 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:14.821517 18503 raft_consensus.cc:2749] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:14.826151 18526 leader_election.cc:290] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:14.827316 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:14.827934 18455 raft_consensus.cc:3060] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.828202 18347 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:14.828490 18307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d115840e17448ae802e5efcbbb29590" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:14.829061 18528 raft_consensus.cc:2804] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.829367 18307 raft_consensus.cc:3060] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:14.829614 18528 raft_consensus.cc:697] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:14.830612 18528 consensus_queue.cc:237] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.836086 18455 raft_consensus.cc:2468] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.837826 18346 leader_election.cc:304] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:14.838212 18307 raft_consensus.cc:2468] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:14.838773 18526 raft_consensus.cc:2804] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:14.839294 18526 raft_consensus.cc:697] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:14.840296 18526 consensus_queue.cc:237] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:14.846689 18203 catalog_manager.cc:5697] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.854666 18503 consensus_queue.cc:1048] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.863706 18202 catalog_manager.cc:5697] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.866973 18503 consensus_queue.cc:1048] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.978824 18526 consensus_queue.cc:1048] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:14.989378 18526 consensus_queue.cc:1048] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:15.054914 18547 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:15.063964 18381 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:15.072824 18553 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:15.074143 18553 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:15.086249 18547 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:15.100947  7638 ts_manager.cc:295] Set tserver state for 921953afbbc9442ba1be12e26f1091c1 to MAINTENANCE_MODE
I20260709 13:04:15.101727  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 13:04:15.135576 18547 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:15.136596  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:15.137405  7638 tablet_replica.cc:333] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.138032  7638 raft_consensus.cc:2243] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:15.139109  7638 raft_consensus.cc:2272] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.141949  7638 tablet_replica.cc:333] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.143591  7638 raft_consensus.cc:2243] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.144049  7638 raft_consensus.cc:2272] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.146550  7638 tablet_replica.cc:333] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.147193  7638 raft_consensus.cc:2243] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.147681  7638 raft_consensus.cc:2272] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.147677 18547 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:15.150763  7638 tablet_replica.cc:333] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.151552  7638 raft_consensus.cc:2243] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.152020  7638 raft_consensus.cc:2272] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.154886  7638 tablet_replica.cc:333] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.155620  7638 raft_consensus.cc:2243] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:15.157300  7638 raft_consensus.cc:2272] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.159392  7638 tablet_replica.cc:333] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.160133  7638 raft_consensus.cc:2243] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:15.160920  7638 raft_consensus.cc:2272] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.163224  7638 tablet_replica.cc:333] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.163769  7638 raft_consensus.cc:2243] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.164182  7638 raft_consensus.cc:2272] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.166651  7638 tablet_replica.cc:333] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.167346  7638 raft_consensus.cc:2243] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.167790  7638 raft_consensus.cc:2272] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.170207  7638 tablet_replica.cc:333] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.170881  7638 raft_consensus.cc:2243] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.171327  7638 raft_consensus.cc:2272] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.174053  7638 tablet_replica.cc:333] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: stopping tablet replica
I20260709 13:04:15.174476  7638 raft_consensus.cc:2243] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:15.174835  7638 raft_consensus.cc:2272] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:15.182236 18503 consensus_queue.cc:1048] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:15.191509 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:15.201808 18421 consensus_peers.cc:597] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:15.210186  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
I20260709 13:04:15.247769 18526 consensus_queue.cc:1048] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:15.265420 18347 consensus_peers.cc:597] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:15.273721 18347 consensus_peers.cc:597] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:15.293278 18347 consensus_peers.cc:597] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:15.341485 18421 consensus_peers.cc:597] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:15.370548 18347 consensus_peers.cc:597] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:15.408697 18528 consensus_queue.cc:1048] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:15.463267 18347 consensus_peers.cc:597] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:16.192516 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:16.432825 18347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111) [suppressed 151 similar messages]
I20260709 13:04:16.564411 18529 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.564962 18529 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.566749 18529 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:16.568504 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
W20260709 13:04:16.570688 18421 leader_election.cc:336] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.571086 18421 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:16.571695 18529 raft_consensus.cc:2749] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.576469 18564 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.576930 18564 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.578830 18564 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:16.579928 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:16.580518 18455 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:16.581707 18346 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:16.582496 18564 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:16.582497 18347 leader_election.cc:336] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.582994 18564 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.583406 18564 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.585667 18588 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.586223 18588 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.588032 18588 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:16.589267 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:16.589998 18455 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 1.
I20260709 13:04:16.590238 18564 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.591114 18346 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:16.591902 18588 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:16.592114 18347 leader_election.cc:336] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.592279 18564 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:16.592294 18588 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.593073 18588 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.593572 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:16.594190 18455 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:16.597188 18347 leader_election.cc:336] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.599647 18588 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.600792 18455 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 2.
I20260709 13:04:16.601718 18588 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:16.601899 18346 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:16.602741 18588 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.603245 18588 raft_consensus.cc:697] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:16.603847 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:16.604460 18455 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.604120 18588 consensus_queue.cc:237] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
W20260709 13:04:16.606175 18347 leader_election.cc:336] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.611037 18455 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 2.
I20260709 13:04:16.612183 18346 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:16.612973 18588 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.613489 18588 raft_consensus.cc:697] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:16.614171 18588 consensus_queue.cc:237] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.614454 18202 catalog_manager.cc:5697] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 1 to 2, leader changed from 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.630519 18202 catalog_manager.cc:5697] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 1 to 2, leader changed from 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.645512 18588 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.646018 18588 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.647666 18588 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:16.648811 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bd62cdab312c454fb693d772d5305ec4" is_pre_election: true
I20260709 13:04:16.650969 18529 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
W20260709 13:04:16.651654 18347 leader_election.cc:336] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.651469 18529 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.652151 18347 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10; no voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:16.652882 18588 raft_consensus.cc:2749] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.653633 18529 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:16.654896 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10" is_pre_election: true
I20260709 13:04:16.655437 18380 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 1.
I20260709 13:04:16.656447 18421 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:16.657164 18529 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:16.657169 18421 leader_election.cc:336] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.657578 18529 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921953afbbc9442ba1be12e26f1091c1)
I20260709 13:04:16.657980 18529 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.663507 18529 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.665071 18529 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:16.666591 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:16.667121 18380 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:16.668643 18421 leader_election.cc:336] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111)
I20260709 13:04:16.671939 18380 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:16.672968 18421 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 921953afbbc9442ba1be12e26f1091c1
I20260709 13:04:16.673753 18529 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.674801 18529 raft_consensus.cc:697] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:16.675525 18529 consensus_queue.cc:237] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:16.682726 18202 catalog_manager.cc:5697] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 1 to 2, leader changed from 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.152665 18455 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.152935 18456 raft_consensus.cc:1275] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.153893 18588 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:17.154284 18597 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:17.167469 18347 consensus_peers.cc:597] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:17.174055 18347 consensus_peers.cc:597] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:17.176630 18380 raft_consensus.cc:1275] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.178107 18529 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:17.191982 18421 consensus_peers.cc:597] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:17.193675 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:17.644019 18347 consensus_peers.cc:597] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:17.789179 18421 consensus_peers.cc:597] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:17.837073 18421 consensus_peers.cc:597] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:17.965647 18347 consensus_peers.cc:597] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:17.992074 18347 consensus_peers.cc:597] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:18.016753 18347 consensus_peers.cc:597] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:18.183952 18347 consensus_peers.cc:597] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:18.195380 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:18.352710 18236 debug-util.cc:398] Leaking SignalData structure 0x7b0800065960 after lost signal to thread 7641
W20260709 13:04:18.354236 18236 debug-util.cc:398] Leaking SignalData structure 0x7b0800052080 after lost signal to thread 18239
W20260709 13:04:18.356047 18236 debug-util.cc:398] Leaking SignalData structure 0x7b08001c43e0 after lost signal to thread 18406
W20260709 13:04:18.357916 18236 debug-util.cc:398] Leaking SignalData structure 0x7b0800246d00 after lost signal to thread 18480
W20260709 13:04:19.196610 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:19.611260 18347 consensus_peers.cc:597] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:19.755618 18347 consensus_peers.cc:597] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:19.798094 18421 consensus_peers.cc:597] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:19.951719 18347 consensus_peers.cc:597] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:20.197810 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:20.206041 18421 consensus_peers.cc:597] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:20.244892  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:20.253333 18608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.256642 18609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:20.258643 18611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:20.264868  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:20.266983  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:20.267318  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:20.267554  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602260267537 us; error 0 us; skew 500 ppm
I20260709 13:04:20.268431  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:20.272079  7638 webserver.cc:533] Webserver started at http://127.7.117.129:39671/ using document root <none> and password file <none>
I20260709 13:04:20.272874  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:20.273118  7638 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:04:20.278810  7638 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:20.284713 18616 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:20.285741  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:20.286181  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "921953afbbc9442ba1be12e26f1091c1"
format_stamp: "Formatted at 2026-07-09 13:04:13 on dist-test-slave-ndzg"
I20260709 13:04:20.286813  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:20.312000  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:20.313930  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:20.315940  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:20.321504 18623 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:04:20.349043 18347 consensus_peers.cc:597] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:20.377599 18421 consensus_peers.cc:597] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:20.378086  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:04:20.378479  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.002s	sys 0.002s
I20260709 13:04:20.378831  7638 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:04:20.384631 18623 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.401506 18623 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.402377 18623 tablet_bootstrap.cc:492] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.403069 18623 ts_tablet_manager.cc:1403] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.002s
I20260709 13:04:20.406267 18623 raft_consensus.cc:359] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.407362 18623 raft_consensus.cc:740] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.408185 18623 consensus_queue.cc:260] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.411285 18623 ts_tablet_manager.cc:1434] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.009s
I20260709 13:04:20.412130 18623 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
W20260709 13:04:20.425961 18347 consensus_peers.cc:597] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:20.429468 18623 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.430387 18623 tablet_bootstrap.cc:492] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.431080 18623 ts_tablet_manager.cc:1403] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.002s
I20260709 13:04:20.433897 18623 raft_consensus.cc:359] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.434636 18623 raft_consensus.cc:740] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.435431 18623 consensus_queue.cc:260] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.437592 18623 ts_tablet_manager.cc:1434] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 13:04:20.438457 18623 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.439949  7638 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:04:20.440336  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.062s	user 0.045s	sys 0.014s
I20260709 13:04:20.459051 18623 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.460259 18623 tablet_bootstrap.cc:492] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.461062 18623 ts_tablet_manager.cc:1403] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 13:04:20.464113 18623 raft_consensus.cc:359] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.464939 18623 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.465931 18623 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.468552 18623 ts_tablet_manager.cc:1434] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:04:20.469588 18623 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.487543 18623 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.488545 18623 tablet_bootstrap.cc:492] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.489355 18623 ts_tablet_manager.cc:1403] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:04:20.492326 18623 raft_consensus.cc:359] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.493124 18623 raft_consensus.cc:740] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.494065 18623 consensus_queue.cc:260] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.496490 18623 ts_tablet_manager.cc:1434] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:20.497442 18623 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.516131 18623 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.517302 18623 tablet_bootstrap.cc:492] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.518064 18623 ts_tablet_manager.cc:1403] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:04:20.521114 18623 raft_consensus.cc:359] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.521966 18623 raft_consensus.cc:740] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.522948 18623 consensus_queue.cc:260] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.525442 18623 ts_tablet_manager.cc:1434] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:20.526348 18623 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.530737  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:46283
I20260709 13:04:20.530965 18685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:46283 every 8 connection(s)
I20260709 13:04:20.549742 18623 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.550971 18623 tablet_bootstrap.cc:492] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.551753 18623 ts_tablet_manager.cc:1403] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 13:04:20.554621 18623 raft_consensus.cc:359] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.555409 18623 raft_consensus.cc:740] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.556749  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.556470 18623 consensus_queue.cc:260] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.559181 18623 ts_tablet_manager.cc:1434] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:04:20.560135 18623 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.590193 18623 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.591234 18623 tablet_bootstrap.cc:492] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.591928 18623 ts_tablet_manager.cc:1403] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.011s
I20260709 13:04:20.596840 18660 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.608106 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:20.609107 18380 raft_consensus.cc:606] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:20.613318 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:20.613458 18623 raft_consensus.cc:359] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:20.614001 18380 raft_consensus.cc:606] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:20.614286 18623 raft_consensus.cc:740] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.615165 18623 consensus_queue.cc:260] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:20.617530 18623 ts_tablet_manager.cc:1434] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:04:20.617643 18687 heartbeater.cc:344] Connected to a master server at 127.7.117.190:35257
I20260709 13:04:20.618130 18687 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:20.618453 18623 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.619158 18687 heartbeater.cc:507] Master 127.7.117.190:35257 requested a full tablet report, sending...
I20260709 13:04:20.621431 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:20.622236 18380 raft_consensus.cc:606] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:20.623986  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:04:20.624673  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:20.626260  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.646667 18623 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.647837 18623 tablet_bootstrap.cc:492] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.648576 18623 ts_tablet_manager.cc:1403] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.011s
I20260709 13:04:20.651186 18623 raft_consensus.cc:359] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.651762 18623 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.652397 18623 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.653335 18202 ts_manager.cc:194] Re-registered known tserver with Master: 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283)
I20260709 13:04:20.654007 18623 ts_tablet_manager.cc:1434] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:04:20.654738 18623 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.667045 18623 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.668049 18202 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59788
I20260709 13:04:20.668102 18623 tablet_bootstrap.cc:492] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.670468 18623 ts_tablet_manager.cc:1403] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:04:20.672011 18687 heartbeater.cc:499] Master 127.7.117.190:35257 was elected leader, sending a full tablet report...
I20260709 13:04:20.673215 18623 raft_consensus.cc:359] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.673885 18623 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.674474 18623 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.676493 18623 ts_tablet_manager.cc:1434] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:04:20.677527 18623 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap starting.
I20260709 13:04:20.692236 18623 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:20.693097 18623 tablet_bootstrap.cc:492] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Bootstrap complete.
I20260709 13:04:20.693904 18623 ts_tablet_manager.cc:1403] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:04:20.695751 18623 raft_consensus.cc:359] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.696277 18623 raft_consensus.cc:740] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921953afbbc9442ba1be12e26f1091c1, State: Initialized, Role: FOLLOWER
I20260709 13:04:20.696969 18623 consensus_queue.cc:260] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.699076 18623 ts_tablet_manager.cc:1434] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:20.705476 18660 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.725229 18606 raft_consensus.cc:993] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:20.725796 18606 raft_consensus.cc:1081] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:20.729612 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:20.730327 18456 raft_consensus.cc:493] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:20.730778 18456 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.741863 18456 raft_consensus.cc:515] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.745199 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:20.745980 18660 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.746016 18456 leader_election.cc:290] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:20.747346 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d213c150e3ee4c2db44dcd864c8e15ab" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:20.747988 18380 raft_consensus.cc:3055] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:20.748328 18380 raft_consensus.cc:740] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:20.749356 18380 consensus_queue.cc:260] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.750798 18380 raft_consensus.cc:3060] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.753947 18660 raft_consensus.cc:2468] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:20.755650 18421 leader_election.cc:304] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:20.756649 18595 raft_consensus.cc:2804] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:20.768303 18595 raft_consensus.cc:697] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:20.770109 18595 consensus_queue.cc:237] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:20.772898 18380 raft_consensus.cc:2468] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:20.779027 18202 catalog_manager.cc:5697] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 1 to 2, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.027274 18660 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.038345 18714 raft_consensus.cc:993] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:21.039137 18606 raft_consensus.cc:1081] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:21.042332 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:21.043078 18456 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.043582 18456 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.051852 18456 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.054754 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:21.055428 18660 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.055526 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:21.056262 18380 raft_consensus.cc:3055] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.056572 18380 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:21.057396 18456 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 3 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:21.057478 18380 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.058794 18380 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.068950 18380 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 3.
I20260709 13:04:21.068991 18660 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 3.
I20260709 13:04:21.070097 18421 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:21.070779 18595 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.071313 18595 raft_consensus.cc:697] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:21.072348 18595 consensus_queue.cc:237] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.081101 18202 catalog_manager.cc:5697] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 2 to 3, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.179984 18380 raft_consensus.cc:1275] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.181376 18595 consensus_queue.cc:1048] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:21.198798 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:21.201437 18660 raft_consensus.cc:1275] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.203125 18595 consensus_queue.cc:1048] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:21.215528 18715 raft_consensus.cc:993] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:21.216193 18703 raft_consensus.cc:1081] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:21.217883 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:21.218487 18456 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.218916 18456 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.227581 18456 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.229905 18456 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 3 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:21.230597 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:21.230885 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:21.231353 18660 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.231508 18380 raft_consensus.cc:3055] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.231984 18380 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:21.232993 18380 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.234081 18380 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.239331 18380 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 3.
I20260709 13:04:21.239820 18660 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 3.
I20260709 13:04:21.240475 18421 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:21.241344 18595 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.241914 18595 raft_consensus.cc:697] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:21.243057 18595 consensus_queue.cc:237] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.252266 18201 catalog_manager.cc:5697] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 2 to 3, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.491431 18380 raft_consensus.cc:1275] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.492592 18595 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.503026 18660 raft_consensus.cc:1275] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.509407 18595 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.639006  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:21.655294 18456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
mode: GRACEFUL
new_leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:04:21.655838 18456 raft_consensus.cc:606] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Received request to transfer leadership to TS 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:21.658994 18456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
mode: GRACEFUL
new_leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:04:21.659658 18456 raft_consensus.cc:606] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Received request to transfer leadership to TS 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:21.661069  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:21.662168  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:21.662803  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:21.670421 18659 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.673202 18705 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.684763 18380 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.687564 18705 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.695751 18706 raft_consensus.cc:993] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: : Instructing follower 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:21.696245 18705 raft_consensus.cc:1081] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Signalling peer 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:21.698966 18380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58044
I20260709 13:04:21.699549 18380 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.699949 18380 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.706614 18380 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.709445 18380 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 4 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:21.710108 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:21.710626 18456 raft_consensus.cc:3055] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:21.710834 18456 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:21.711498 18456 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.712272 18456 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.714205 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:21.714844 18660 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.719959 18456 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 4.
I20260709 13:04:21.721477 18346 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d29f4c02f0400f8f67c124f88b8c10, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:21.722223 18703 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.722703 18703 raft_consensus.cc:697] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:21.723469 18703 consensus_queue.cc:237] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:21.725669 18660 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 4.
I20260709 13:04:21.730520 18202 catalog_manager.cc:5697] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 3 to 4, leader changed from bd62cdab312c454fb693d772d5305ec4 (127.7.117.131) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.019878 18706 raft_consensus.cc:993] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: : Instructing follower 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:22.020370 18705 raft_consensus.cc:1081] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Signalling peer 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:22.021831 18380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58044
I20260709 13:04:22.022269 18380 raft_consensus.cc:493] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.022528 18380 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.027508 18380 raft_consensus.cc:515] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.028859 18380 leader_election.cc:290] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 4 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:22.029691 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:22.029935 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fc9957cf49e41f7b735627dd4bef580" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:22.030367 18660 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.030509 18456 raft_consensus.cc:3055] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:22.030915 18456 raft_consensus.cc:740] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:22.031741 18456 consensus_queue.cc:260] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.033058 18456 raft_consensus.cc:3060] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.037806 18660 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 4.
I20260709 13:04:22.039156 18347 leader_election.cc:304] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:22.039865 18703 raft_consensus.cc:2804] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:22.040161 18456 raft_consensus.cc:2468] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 4.
I20260709 13:04:22.040374 18703 raft_consensus.cc:697] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:22.041298 18703 consensus_queue.cc:237] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.047317 18202 catalog_manager.cc:5697] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 3 to 4, leader changed from bd62cdab312c454fb693d772d5305ec4 (127.7.117.131) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 4 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.187783 18456 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:22.188944 18703 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:22.198339 18659 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:22.199877 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:22.202731 18703 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:22.581485 18456 raft_consensus.cc:1275] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:22.582906 18728 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:22.592203 18659 raft_consensus.cc:1275] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:22.593978 18715 consensus_queue.cc:1048] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:22.669560  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:22.676746 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2386e84784c94cae9251411d38311a1f"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:22.677258 18380 raft_consensus.cc:606] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:22.679921 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:22.680446 18380 raft_consensus.cc:606] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:22.681632  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:22.682160  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:22.682729  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:22.815454 18703 raft_consensus.cc:993] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:22.815987 18728 raft_consensus.cc:1081] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:22.817626 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:22.818186 18456 raft_consensus.cc:493] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.818559 18456 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.823805 18456 raft_consensus.cc:515] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.825246 18456 leader_election.cc:290] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:22.826110 18659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:22.826431 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4726c89fa9384720bf59d96ac827e73b" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:22.826788 18659 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.827112 18380 raft_consensus.cc:3055] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.827468 18380 raft_consensus.cc:740] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:22.828413 18380 consensus_queue.cc:260] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.829803 18380 raft_consensus.cc:3060] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.832993 18659 raft_consensus.cc:2468] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:22.834126 18421 leader_election.cc:304] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:22.835302 18705 raft_consensus.cc:2804] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.836119 18705 raft_consensus.cc:697] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:22.837347 18705 consensus_queue.cc:237] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:22.835220 18380 raft_consensus.cc:2468] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:22.846026 18202 catalog_manager.cc:5697] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 1 to 2, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.022298 18715 raft_consensus.cc:993] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:23.022797 18728 raft_consensus.cc:1081] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:23.024140 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2386e84784c94cae9251411d38311a1f"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:23.024633 18456 raft_consensus.cc:493] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.024957 18456 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.030076 18456 raft_consensus.cc:515] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:23.031582 18456 leader_election.cc:290] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:23.032406 18659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:23.032578 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2386e84784c94cae9251411d38311a1f" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:23.033104 18659 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.033160 18380 raft_consensus.cc:3055] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.033622 18380 raft_consensus.cc:740] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:23.034415 18380 consensus_queue.cc:260] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:23.035629 18380 raft_consensus.cc:3060] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.040140 18659 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:23.041419 18421 leader_election.cc:304] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:23.042038 18380 raft_consensus.cc:2468] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 2.
I20260709 13:04:23.042094 18735 raft_consensus.cc:2804] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.042658 18735 raft_consensus.cc:697] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:23.043591 18735 consensus_queue.cc:237] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:23.050359 18202 catalog_manager.cc:5697] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 1 to 2, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:23.201052 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:23.245659 18380 raft_consensus.cc:1275] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.247061 18735 consensus_queue.cc:1048] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.258230 18659 raft_consensus.cc:1275] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.260607 18705 consensus_queue.cc:1048] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.430830 18659 raft_consensus.cc:1275] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.432081 18705 consensus_queue.cc:1048] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.440322 18380 raft_consensus.cc:1275] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.441622 18734 consensus_queue.cc:1048] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.690001  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.697141 18456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
mode: GRACEFUL
new_leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:04:23.697750 18456 raft_consensus.cc:606] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Received request to transfer leadership to TS 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:23.700234 18456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
mode: GRACEFUL
new_leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:04:23.700701 18456 raft_consensus.cc:606] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Received request to transfer leadership to TS 92d29f4c02f0400f8f67c124f88b8c10
I20260709 13:04:23.701752  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:23.702273  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:23.702849  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.991693 18705 raft_consensus.cc:993] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: : Instructing follower 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:23.992251 18735 raft_consensus.cc:1081] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Signalling peer 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:23.993760 18380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58044
I20260709 13:04:23.994176 18380 raft_consensus.cc:493] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.994412 18380 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.000458 18380 raft_consensus.cc:515] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:24.002578 18380 leader_election.cc:290] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Requested vote from peers bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683), 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283)
I20260709 13:04:24.002839 18456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:24.003191 18659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0e6123188d48399753b8317fd4213e" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:24.003520 18456 raft_consensus.cc:3055] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:24.003768 18659 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.003829 18456 raft_consensus.cc:740] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:24.005092 18456 consensus_queue.cc:260] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:24.006486 18456 raft_consensus.cc:3060] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.011134 18659 raft_consensus.cc:2468] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 2.
I20260709 13:04:24.012441 18347 leader_election.cc:304] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:24.013324 18749 raft_consensus.cc:2804] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:24.013434 18456 raft_consensus.cc:2468] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 2.
I20260709 13:04:24.013856 18749 raft_consensus.cc:697] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:24.014894 18749 consensus_queue.cc:237] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } }
I20260709 13:04:24.022029 18201 catalog_manager.cc:5697] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 1 to 2, leader changed from bd62cdab312c454fb693d772d5305ec4 (127.7.117.131) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.080627 18705 raft_consensus.cc:993] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: : Instructing follower 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:24.081193 18735 raft_consensus.cc:1081] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Signalling peer 92d29f4c02f0400f8f67c124f88b8c10 to start an election
I20260709 13:04:24.082930 18380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
 from {username='slave'} at 127.0.0.1:58044
I20260709 13:04:24.083384 18380 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:24.083643 18380 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.088924 18380 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.090451 18380 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 3 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), bd62cdab312c454fb693d772d5305ec4 (127.7.117.131:36683)
I20260709 13:04:24.091239 18659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:24.091660 18455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "92d29f4c02f0400f8f67c124f88b8c10" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
I20260709 13:04:24.091805 18659 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.092258 18455 raft_consensus.cc:3055] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:24.092559 18455 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:24.093247 18455 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.094250 18455 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.097249 18659 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 3.
I20260709 13:04:24.098450 18347 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, 92d29f4c02f0400f8f67c124f88b8c10; no voters: 
I20260709 13:04:24.099115 18749 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:24.099538 18749 raft_consensus.cc:697] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 LEADER]: Becoming Leader. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:24.099642 18455 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d29f4c02f0400f8f67c124f88b8c10 in term 3.
I20260709 13:04:24.100442 18749 consensus_queue.cc:237] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.107921 18201 catalog_manager.cc:5697] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 reported cstate change: term changed from 2 to 3, leader changed from bd62cdab312c454fb693d772d5305ec4 (127.7.117.131) to 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130). New cstate: current_term: 3 leader_uuid: "92d29f4c02f0400f8f67c124f88b8c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:24.202088 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:24.394183 18660 raft_consensus.cc:1275] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.395431 18749 consensus_queue.cc:1048] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.412164 18455 raft_consensus.cc:1275] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.414098 18759 consensus_queue.cc:1048] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.497430 18660 raft_consensus.cc:1275] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:24.498725 18759 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:24.509536 18455 raft_consensus.cc:1275] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Refusing update from remote peer 92d29f4c02f0400f8f67c124f88b8c10: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:24.510686 18759 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:24.709187  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.716626 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:24.717134 18380 raft_consensus.cc:606] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:24.719843 18380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265"
dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
mode: GRACEFUL
new_leader_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58050
I20260709 13:04:24.720326 18380 raft_consensus.cc:606] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 LEADER]: Received request to transfer leadership to TS bd62cdab312c454fb693d772d5305ec4
I20260709 13:04:24.721376  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:24.721899  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:24.722488  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.763358 18728 raft_consensus.cc:993] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:24.763914 18728 raft_consensus.cc:1081] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:24.765413 18455 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:24.765878 18455 raft_consensus.cc:493] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:24.766181 18455 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.771282 18455 raft_consensus.cc:515] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.772640 18455 leader_election.cc:290] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 5 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:24.773566 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:24.773774 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b62fc33c925046378913d49257ecfbe7" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:24.774225 18660 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.774413 18380 raft_consensus.cc:3055] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:24.774818 18380 raft_consensus.cc:740] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:24.775470 18380 consensus_queue.cc:260] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.776382 18380 raft_consensus.cc:3060] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.780727 18660 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 5.
I20260709 13:04:24.781353 18380 raft_consensus.cc:2468] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 5.
I20260709 13:04:24.781891 18421 leader_election.cc:304] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:24.782577 18735 raft_consensus.cc:2804] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:24.783099 18735 raft_consensus.cc:697] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 5 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:24.783855 18735 consensus_queue.cc:237] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:24.791007 18201 catalog_manager.cc:5697] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 4 to 5, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 5 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.131356 18759 raft_consensus.cc:993] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: : Instructing follower bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:25.131860 18728 raft_consensus.cc:1081] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 LEADER]: Signalling peer bd62cdab312c454fb693d772d5305ec4 to start an election
I20260709 13:04:25.133476 18456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265"
dest_uuid: "bd62cdab312c454fb693d772d5305ec4"
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:04:25.133972 18456 raft_consensus.cc:493] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:25.134369 18456 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.138715 18456 raft_consensus.cc:515] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:25.140262 18456 leader_election.cc:290] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 4 election: Requested vote from peers 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283), 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213)
I20260709 13:04:25.141006 18660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "921953afbbc9442ba1be12e26f1091c1"
I20260709 13:04:25.141188 18380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf45dde5c5a4bb8ab999adb039a8265" candidate_uuid: "bd62cdab312c454fb693d772d5305ec4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "92d29f4c02f0400f8f67c124f88b8c10"
I20260709 13:04:25.141731 18660 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.141772 18380 raft_consensus.cc:3055] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:25.142143 18380 raft_consensus.cc:740] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92d29f4c02f0400f8f67c124f88b8c10, State: Running, Role: LEADER
I20260709 13:04:25.142760 18380 consensus_queue.cc:260] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:25.143695 18380 raft_consensus.cc:3060] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.146826 18660 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 4.
I20260709 13:04:25.147984 18421 leader_election.cc:304] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 921953afbbc9442ba1be12e26f1091c1, bd62cdab312c454fb693d772d5305ec4; no voters: 
I20260709 13:04:25.148030 18380 raft_consensus.cc:2468] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62cdab312c454fb693d772d5305ec4 in term 4.
I20260709 13:04:25.148804 18735 raft_consensus.cc:2804] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:25.149267 18735 raft_consensus.cc:697] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 4 LEADER]: Becoming Leader. State: Replica: bd62cdab312c454fb693d772d5305ec4, State: Running, Role: LEADER
I20260709 13:04:25.149984 18735 consensus_queue.cc:237] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } }
I20260709 13:04:25.156041 18201 catalog_manager.cc:5697] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 reported cstate change: term changed from 3 to 4, leader changed from 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130) to bd62cdab312c454fb693d772d5305ec4 (127.7.117.131). New cstate: current_term: 4 leader_uuid: "bd62cdab312c454fb693d772d5305ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd62cdab312c454fb693d772d5305ec4" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 36683 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.203043 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:25.230254 18380 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 5 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:25.231540 18735 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:25.240407 18660 raft_consensus.cc:1275] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 5 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:25.242234 18735 consensus_queue.cc:1048] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:25.537856 18660 raft_consensus.cc:1275] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:25.539219 18735 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921953afbbc9442ba1be12e26f1091c1" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:25.547641 18380 raft_consensus.cc:1275] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Refusing update from remote peer bd62cdab312c454fb693d772d5305ec4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:25.549707 18705 consensus_queue.cc:1048] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d29f4c02f0400f8f67c124f88b8c10" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:25.814780  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:25.871675  7638 tablet_server.cc:179] TabletServer@127.7.117.129:46283 shutting down...
I20260709 13:04:25.893939  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:25.894574  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.895145  7638 raft_consensus.cc:2243] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.895589  7638 raft_consensus.cc:2272] T b62fc33c925046378913d49257ecfbe7 P 921953afbbc9442ba1be12e26f1091c1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.897847  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.898406  7638 raft_consensus.cc:2243] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.898821  7638 raft_consensus.cc:2272] T 0a0e6123188d48399753b8317fd4213e P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.900614  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.901153  7638 raft_consensus.cc:2243] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.901618  7638 raft_consensus.cc:2272] T d213c150e3ee4c2db44dcd864c8e15ab P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.903329  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.903854  7638 raft_consensus.cc:2243] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.904237  7638 raft_consensus.cc:2272] T 721b78f6d9594c61bf4e11677700dd2a P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.906263  7638 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:25.906773 18421 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 86 similar messages]
I20260709 13:04:25.906932  7638 raft_consensus.cc:2243] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.907586  7638 raft_consensus.cc:2272] T 6d115840e17448ae802e5efcbbb29590 P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.909994  7638 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:25.909901 18421 consensus_peers.cc:597] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.910799  7638 raft_consensus.cc:2243] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.911209  7638 raft_consensus.cc:2272] T dcf45dde5c5a4bb8ab999adb039a8265 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.913934  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.914724  7638 raft_consensus.cc:2243] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.915084  7638 raft_consensus.cc:2272] T 4726c89fa9384720bf59d96ac827e73b P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.916817  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.917351  7638 raft_consensus.cc:2243] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.917727  7638 raft_consensus.cc:2272] T 3fc9957cf49e41f7b735627dd4bef580 P 921953afbbc9442ba1be12e26f1091c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.919413  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.919823  7638 raft_consensus.cc:2243] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.920166  7638 raft_consensus.cc:2272] T b940b75365604200ad40c41249a0e28f P 921953afbbc9442ba1be12e26f1091c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.921861  7638 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.922261  7638 raft_consensus.cc:2243] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.922598  7638 raft_consensus.cc:2272] T 2386e84784c94cae9251411d38311a1f P 921953afbbc9442ba1be12e26f1091c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.951386  7638 tablet_server.cc:196] TabletServer@127.7.117.129:46283 shutdown complete.
W20260709 13:04:25.956743 18421 consensus_peers.cc:597] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.961792 18421 consensus_peers.cc:597] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.967317 18347 consensus_peers.cc:597] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.971153 18421 consensus_peers.cc:597] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.972916  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
I20260709 13:04:25.998232  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:25.999053  7638 tablet_replica.cc:333] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.000267  7638 raft_consensus.cc:2243] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.000962  7638 raft_consensus.cc:2272] T b940b75365604200ad40c41249a0e28f P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.003307  7638 tablet_replica.cc:333] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.003890  7638 raft_consensus.cc:2243] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.004330  7638 raft_consensus.cc:2272] T 2386e84784c94cae9251411d38311a1f P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.006655  7638 tablet_replica.cc:333] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.007138  7638 raft_consensus.cc:2243] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.007484  7638 raft_consensus.cc:2272] T b62fc33c925046378913d49257ecfbe7 P 92d29f4c02f0400f8f67c124f88b8c10 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.010407  7638 tablet_replica.cc:333] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.010908  7638 raft_consensus.cc:2243] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.011981  7638 raft_consensus.cc:2272] T 3fc9957cf49e41f7b735627dd4bef580 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.015312  7638 tablet_replica.cc:333] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.015934  7638 raft_consensus.cc:2243] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.016758  7638 raft_consensus.cc:2272] T 0a0e6123188d48399753b8317fd4213e P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.019093  7638 tablet_replica.cc:333] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.019675  7638 raft_consensus.cc:2243] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.020095  7638 raft_consensus.cc:2272] T d213c150e3ee4c2db44dcd864c8e15ab P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.022322  7638 tablet_replica.cc:333] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.022868  7638 raft_consensus.cc:2243] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.023658  7638 raft_consensus.cc:2272] T 6d115840e17448ae802e5efcbbb29590 P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.025761  7638 tablet_replica.cc:333] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.026315  7638 raft_consensus.cc:2243] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.026721  7638 raft_consensus.cc:2272] T 4726c89fa9384720bf59d96ac827e73b P 92d29f4c02f0400f8f67c124f88b8c10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.028746  7638 tablet_replica.cc:333] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.029211  7638 raft_consensus.cc:2243] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.029659  7638 raft_consensus.cc:2272] T dcf45dde5c5a4bb8ab999adb039a8265 P 92d29f4c02f0400f8f67c124f88b8c10 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.032011  7638 tablet_replica.cc:333] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10: stopping tablet replica
I20260709 13:04:26.032513  7638 raft_consensus.cc:2243] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.033367  7638 raft_consensus.cc:2272] T 721b78f6d9594c61bf4e11677700dd2a P 92d29f4c02f0400f8f67c124f88b8c10 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.049603 18421 consensus_peers.cc:597] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 -> Peer 921953afbbc9442ba1be12e26f1091c1 (127.7.117.129:46283): Couldn't send request to peer 921953afbbc9442ba1be12e26f1091c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:46283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.058513  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
W20260709 13:04:26.068832 18421 consensus_peers.cc:597] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 -> Peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Couldn't send request to peer 92d29f4c02f0400f8f67c124f88b8c10. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:41213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.085389  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
W20260709 13:04:26.092680 18421 consensus_peers.cc:597] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 -> Peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Couldn't send request to peer 92d29f4c02f0400f8f67c124f88b8c10. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:41213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.116104  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:26.117069  7638 tablet_replica.cc:333] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.117998  7638 raft_consensus.cc:2243] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.118530  7638 raft_consensus.cc:2272] T 721b78f6d9594c61bf4e11677700dd2a P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.120137 18421 consensus_peers.cc:597] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 -> Peer 92d29f4c02f0400f8f67c124f88b8c10 (127.7.117.130:41213): Couldn't send request to peer 92d29f4c02f0400f8f67c124f88b8c10. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:41213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.121222  7638 tablet_replica.cc:333] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.121893  7638 raft_consensus.cc:2243] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.122320  7638 raft_consensus.cc:2272] T 6d115840e17448ae802e5efcbbb29590 P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.124365  7638 tablet_replica.cc:333] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.124863  7638 raft_consensus.cc:2243] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.125598  7638 raft_consensus.cc:2272] T 4726c89fa9384720bf59d96ac827e73b P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.127583  7638 tablet_replica.cc:333] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.128120  7638 raft_consensus.cc:2243] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.128808  7638 raft_consensus.cc:2272] T dcf45dde5c5a4bb8ab999adb039a8265 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.131937  7638 tablet_replica.cc:333] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.132469  7638 raft_consensus.cc:2243] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.133149  7638 raft_consensus.cc:2272] T 2386e84784c94cae9251411d38311a1f P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.135126  7638 tablet_replica.cc:333] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.135679  7638 raft_consensus.cc:2243] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.136391  7638 raft_consensus.cc:2272] T b940b75365604200ad40c41249a0e28f P bd62cdab312c454fb693d772d5305ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.138397  7638 tablet_replica.cc:333] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.138850  7638 raft_consensus.cc:2243] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.139542  7638 raft_consensus.cc:2272] T b62fc33c925046378913d49257ecfbe7 P bd62cdab312c454fb693d772d5305ec4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.141913  7638 tablet_replica.cc:333] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.142453  7638 raft_consensus.cc:2243] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.142867  7638 raft_consensus.cc:2272] T 3fc9957cf49e41f7b735627dd4bef580 P bd62cdab312c454fb693d772d5305ec4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.144713  7638 tablet_replica.cc:333] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.145215  7638 raft_consensus.cc:2243] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.146004  7638 raft_consensus.cc:2272] T d213c150e3ee4c2db44dcd864c8e15ab P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.147910  7638 tablet_replica.cc:333] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4: stopping tablet replica
I20260709 13:04:26.148409  7638 raft_consensus.cc:2243] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.148794  7638 raft_consensus.cc:2272] T 0a0e6123188d48399753b8317fd4213e P bd62cdab312c454fb693d772d5305ec4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.204041 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:26.795264 18477 debug-util.cc:398] Leaking SignalData structure 0x7b08001c3bc0 after lost signal to thread 7641
W20260709 13:04:26.796375 18477 debug-util.cc:398] Leaking SignalData structure 0x7b080021a3c0 after lost signal to thread 18239
W20260709 13:04:27.177277  7638 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18477)
W20260709 13:04:27.205081 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:28.178215  7638 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 18477)
W20260709 13:04:28.207135 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:29.178943  7638 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 18477)
W20260709 13:04:29.208503 18253 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:29.680191  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 13:04:29.722606  7638 master.cc:562] Master@127.7.117.190:35257 shutting down...
I20260709 13:04:29.748922  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:29.749917  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.750351  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P 91590c962bcb4bca8505ba0b2e81fd9e: stopping tablet replica
I20260709 13:04:29.790499  7638 master.cc:584] Master@127.7.117.190:35257 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (16779 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:04:29.842046  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:40945
I20260709 13:04:29.843204  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:29.848888 18787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:29.848956 18786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:29.850246 18789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:29.851348  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:29.852217  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:29.852394  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:29.852502  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602269852492 us; error 0 us; skew 500 ppm
I20260709 13:04:29.852958  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:29.855460  7638 webserver.cc:533] Webserver started at http://127.7.117.190:42307/ using document root <none> and password file <none>
I20260709 13:04:29.855947  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:29.856106  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:29.856320  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:29.857923  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "9eedf9c4f928455d903141210ff80f39"
format_stamp: "Formatted at 2026-07-09 13:04:29 on dist-test-slave-ndzg"
I20260709 13:04:29.862470  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:29.865990 18794 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:29.866787  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:04:29.867089  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "9eedf9c4f928455d903141210ff80f39"
format_stamp: "Formatted at 2026-07-09 13:04:29 on dist-test-slave-ndzg"
I20260709 13:04:29.867398  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:29.878849  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:29.880064  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:29.918936  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:40945
I20260709 13:04:29.919059 18845 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:40945 every 8 connection(s)
I20260709 13:04:29.923260 18846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:29.935819 18846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39: Bootstrap starting.
I20260709 13:04:29.940853 18846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.945140 18846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39: No bootstrap required, opened a new log
I20260709 13:04:29.947373 18846 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER }
I20260709 13:04:29.947798 18846 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.948081 18846 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eedf9c4f928455d903141210ff80f39, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.948686 18846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER }
I20260709 13:04:29.949168 18846 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:29.949432 18846 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:29.949721 18846 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.954738 18846 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER }
I20260709 13:04:29.955281 18846 leader_election.cc:304] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eedf9c4f928455d903141210ff80f39; no voters: 
I20260709 13:04:29.956466 18846 leader_election.cc:290] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:29.956830 18849 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:29.958420 18849 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 1 LEADER]: Becoming Leader. State: Replica: 9eedf9c4f928455d903141210ff80f39, State: Running, Role: LEADER
I20260709 13:04:29.959154 18849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER }
I20260709 13:04:29.959517 18846 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:29.965597 18850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eedf9c4f928455d903141210ff80f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER } }
I20260709 13:04:29.966298 18850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:29.966298 18851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eedf9c4f928455d903141210ff80f39. Latest consensus state: current_term: 1 leader_uuid: "9eedf9c4f928455d903141210ff80f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedf9c4f928455d903141210ff80f39" member_type: VOTER } }
I20260709 13:04:29.967199 18851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:29.969978 18856 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:29.975193 18856 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:29.977097  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:29.984513 18856 catalog_manager.cc:1383] Generated new cluster ID: e8ed25339848429eaea672207c8306d2
I20260709 13:04:29.984812 18856 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:30.037225 18856 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:30.038637 18856 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:30.056591 18856 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39: Generated new TSK 0
I20260709 13:04:30.057220 18856 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:30.108065  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:30.114318 18867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:30.115171 18868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:30.118346 18870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:30.120090  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:30.121006  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:30.121206  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:30.121420  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602270121401 us; error 0 us; skew 500 ppm
I20260709 13:04:30.121924  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:30.124104  7638 webserver.cc:533] Webserver started at http://127.7.117.129:45293/ using document root <none> and password file <none>
I20260709 13:04:30.124536  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:30.124686  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:30.124890  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:30.126008  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "35a74035155c4a4cb1a0541d5966f32b"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.130366  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:04:30.133827 18875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.134616  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260709 13:04:30.134948  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "35a74035155c4a4cb1a0541d5966f32b"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.135270  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:30.152551  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:30.153862  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:30.155368  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:30.157902  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:30.158130  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.158367  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:30.158545  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.200126  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:45963
I20260709 13:04:30.200237 18937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:45963 every 8 connection(s)
I20260709 13:04:30.205374  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:30.214917 18942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:30.215839 18943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:30.216301  7638 server_base.cc:1061] running on GCE node
W20260709 13:04:30.216964 18945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:30.218161  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:30.218441  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:30.218636  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602270218619 us; error 0 us; skew 500 ppm
I20260709 13:04:30.218974 18938 heartbeater.cc:344] Connected to a master server at 127.7.117.190:40945
I20260709 13:04:30.219364  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:30.219386 18938 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:30.220492 18938 heartbeater.cc:507] Master 127.7.117.190:40945 requested a full tablet report, sending...
I20260709 13:04:30.222594  7638 webserver.cc:533] Webserver started at http://127.7.117.130:46771/ using document root <none> and password file <none>
I20260709 13:04:30.222769 18811 ts_manager.cc:194] Registered new tserver with Master: 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963)
I20260709 13:04:30.223222  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:30.223498  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:30.223762  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:30.224852 18811 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59318
I20260709 13:04:30.225034  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "74c46aafd0084285b93ed5785e30497d"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.229851  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:30.233280 18950 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.234097  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:30.234401  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "74c46aafd0084285b93ed5785e30497d"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.234665  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:30.250695  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:30.251881  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:30.253346  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:30.255537  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:30.255755  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.256021  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:30.256214  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.297974  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:42477
I20260709 13:04:30.298080 19012 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:42477 every 8 connection(s)
I20260709 13:04:30.302982  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:30.310628 19016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:30.312449 19017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:30.315894 19019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:30.316416  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:30.317101 19013 heartbeater.cc:344] Connected to a master server at 127.7.117.190:40945
I20260709 13:04:30.317561 19013 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:30.317600  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:30.317940  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:30.318118  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602270318085 us; error 0 us; skew 500 ppm
I20260709 13:04:30.318493 19013 heartbeater.cc:507] Master 127.7.117.190:40945 requested a full tablet report, sending...
I20260709 13:04:30.318801  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:30.321018 18811 ts_manager.cc:194] Registered new tserver with Master: 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477)
I20260709 13:04:30.321950  7638 webserver.cc:533] Webserver started at http://127.7.117.131:33337/ using document root <none> and password file <none>
I20260709 13:04:30.322633  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:30.322876  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:30.323014 18811 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59324
I20260709 13:04:30.323199  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:30.324841  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.329450  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:30.332939 19024 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.333878  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:30.334190  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
format_stamp: "Formatted at 2026-07-09 13:04:30 on dist-test-slave-ndzg"
I20260709 13:04:30.334499  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:30.345679  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:30.346982  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:30.348482  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:30.350747  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:30.350983  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.351222  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:30.351395  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:30.393105  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:37675
I20260709 13:04:30.393226 19086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:37675 every 8 connection(s)
I20260709 13:04:30.407835 19087 heartbeater.cc:344] Connected to a master server at 127.7.117.190:40945
I20260709 13:04:30.408255 19087 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:30.408996 19087 heartbeater.cc:507] Master 127.7.117.190:40945 requested a full tablet report, sending...
I20260709 13:04:30.410957 18811 ts_manager.cc:194] Registered new tserver with Master: 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:30.411227  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014685083s
I20260709 13:04:30.412791 18811 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59340
I20260709 13:04:30.438056 18811 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59348:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:30.440243 18811 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:30.489069 18902 tablet_service.cc:1511] Processing CreateTablet for tablet 794bc6906476423296e3ea954ef14ea8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.489046 18903 tablet_service.cc:1511] Processing CreateTablet for tablet 22c965f81d78414380429ba60d0362f8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.491686 18978 tablet_service.cc:1511] Processing CreateTablet for tablet 22c965f81d78414380429ba60d0362f8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.490788 18902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794bc6906476423296e3ea954ef14ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.493172 18978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c965f81d78414380429ba60d0362f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.493110 18977 tablet_service.cc:1511] Processing CreateTablet for tablet 794bc6906476423296e3ea954ef14ea8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.493654 18903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c965f81d78414380429ba60d0362f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.494638 18977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794bc6906476423296e3ea954ef14ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.515000 19102 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: Bootstrap starting.
I20260709 13:04:30.523046 19102 tablet_bootstrap.cc:654] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.523833 19103 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: Bootstrap starting.
I20260709 13:04:30.523068 19052 tablet_service.cc:1511] Processing CreateTablet for tablet 22c965f81d78414380429ba60d0362f8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.526003 19052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c965f81d78414380429ba60d0362f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.524330 19051 tablet_service.cc:1511] Processing CreateTablet for tablet 794bc6906476423296e3ea954ef14ea8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=114ad20fd5e940f2a6484e115cec23f7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.529707 19051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794bc6906476423296e3ea954ef14ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.531101 19102 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: No bootstrap required, opened a new log
I20260709 13:04:30.531525 19103 tablet_bootstrap.cc:654] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.531682 19102 ts_tablet_manager.cc:1403] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:04:30.535418 19102 raft_consensus.cc:359] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.536584 19102 raft_consensus.cc:385] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.536984 19102 raft_consensus.cc:740] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.542282 19102 consensus_queue.cc:260] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.544113 19103 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: No bootstrap required, opened a new log
I20260709 13:04:30.544945 19103 ts_tablet_manager.cc:1403] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
I20260709 13:04:30.549276 19103 raft_consensus.cc:359] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.550491 19103 raft_consensus.cc:385] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.550978 19103 raft_consensus.cc:740] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.551442 19107 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: Bootstrap starting.
I20260709 13:04:30.552254 19103 consensus_queue.cc:260] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.554522 19013 heartbeater.cc:499] Master 127.7.117.190:40945 was elected leader, sending a full tablet report...
I20260709 13:04:30.555673 19102 ts_tablet_manager.cc:1434] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.015s
I20260709 13:04:30.556747 19102 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: Bootstrap starting.
I20260709 13:04:30.561650 19107 tablet_bootstrap.cc:654] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.562880 19103 ts_tablet_manager.cc:1434] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: Time spent starting tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:04:30.563759 18938 heartbeater.cc:499] Master 127.7.117.190:40945 was elected leader, sending a full tablet report...
I20260709 13:04:30.564184 19102 tablet_bootstrap.cc:654] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.567261 19103 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: Bootstrap starting.
I20260709 13:04:30.575475 19103 tablet_bootstrap.cc:654] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.581382 19102 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: No bootstrap required, opened a new log
I20260709 13:04:30.581990 19102 ts_tablet_manager.cc:1403] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.012s
I20260709 13:04:30.584527 19107 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: No bootstrap required, opened a new log
I20260709 13:04:30.584774 19103 tablet_bootstrap.cc:492] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: No bootstrap required, opened a new log
I20260709 13:04:30.585108 19107 ts_tablet_manager.cc:1403] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.018s
I20260709 13:04:30.584771 19102 raft_consensus.cc:359] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.585587 19103 ts_tablet_manager.cc:1403] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.000s
I20260709 13:04:30.585780 19102 raft_consensus.cc:385] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.586170 19102 raft_consensus.cc:740] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.586895 19102 consensus_queue.cc:260] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.588404 19107 raft_consensus.cc:359] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.588642 19103 raft_consensus.cc:359] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.588915 19102 ts_tablet_manager.cc:1434] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:04:30.589329 19107 raft_consensus.cc:385] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.589440 19103 raft_consensus.cc:385] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.589895 19107 raft_consensus.cc:740] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.589998 19103 raft_consensus.cc:740] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.590828 19107 consensus_queue.cc:260] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.591024 19103 consensus_queue.cc:260] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.593678 19103 ts_tablet_manager.cc:1434] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:30.595707 19087 heartbeater.cc:499] Master 127.7.117.190:40945 was elected leader, sending a full tablet report...
I20260709 13:04:30.596437 19107 ts_tablet_manager.cc:1434] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 13:04:30.597530 19107 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: Bootstrap starting.
I20260709 13:04:30.604549 19107 tablet_bootstrap.cc:654] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.609577 19107 tablet_bootstrap.cc:492] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: No bootstrap required, opened a new log
I20260709 13:04:30.610018 19107 ts_tablet_manager.cc:1403] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 13:04:30.612414 19107 raft_consensus.cc:359] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.612982 19107 raft_consensus.cc:385] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.613214 19107 raft_consensus.cc:740] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.613842 19107 consensus_queue.cc:260] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.615763 19107 ts_tablet_manager.cc:1434] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:30.681596 19108 raft_consensus.cc:493] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.682395 19108 raft_consensus.cc:515] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.685158 19108 leader_election.cc:290] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:30.695542 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d" is_pre_election: true
I20260709 13:04:30.696573 18988 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 0.
I20260709 13:04:30.696933 19062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" is_pre_election: true
I20260709 13:04:30.697786 19062 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 0.
I20260709 13:04:30.698170 18878 leader_election.cc:304] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a74035155c4a4cb1a0541d5966f32b, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:30.699026 19108 raft_consensus.cc:2804] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:30.699429 19108 raft_consensus.cc:493] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.699704 19108 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.705791 19108 raft_consensus.cc:515] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.707372 19108 leader_election.cc:290] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 election: Requested vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:30.708308 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:30.708499 19062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:30.708899 18988 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.709224 19062 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.715003 18988 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 1.
I20260709 13:04:30.716053 19062 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 1.
I20260709 13:04:30.716348 18878 leader_election.cc:304] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a74035155c4a4cb1a0541d5966f32b, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:30.717438 19108 raft_consensus.cc:2804] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:30.718515 19108 raft_consensus.cc:697] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Becoming Leader. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:30.719369 19108 consensus_queue.cc:237] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.727172 18811 catalog_manager.cc:5697] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b reported cstate change: term changed from 0 to 1, leader changed from <none> to 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129). New cstate: current_term: 1 leader_uuid: "35a74035155c4a4cb1a0541d5966f32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.780855 19112 raft_consensus.cc:493] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.781491 19112 raft_consensus.cc:515] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.784054 19112 leader_election.cc:290] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963)
I20260709 13:04:30.795435 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d" is_pre_election: true
I20260709 13:04:30.796222 18988 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 0.
I20260709 13:04:30.796815 18913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a74035155c4a4cb1a0541d5966f32b" is_pre_election: true
I20260709 13:04:30.797490 19027 leader_election.cc:304] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:30.797657 18913 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 0.
I20260709 13:04:30.798435 19112 raft_consensus.cc:2804] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:30.798808 19112 raft_consensus.cc:493] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.799139 19112 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.805392 19112 raft_consensus.cc:515] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.807325 19112 leader_election.cc:290] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 1 election: Requested vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963)
I20260709 13:04:30.807942 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:30.808475 18913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
I20260709 13:04:30.808561 18988 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.809088 18913 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:30.814038 18988 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 1.
I20260709 13:04:30.815025 18913 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 1.
I20260709 13:04:30.815217 19027 leader_election.cc:304] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:30.816293 19112 raft_consensus.cc:2804] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:30.817219 19112 raft_consensus.cc:697] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 LEADER]: Becoming Leader. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Running, Role: LEADER
I20260709 13:04:30.818055 19112 consensus_queue.cc:237] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.825430 18811 catalog_manager.cc:5697] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.908473 18811 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59356:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:30.910485 18811 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:30.934852 18902 tablet_service.cc:1511] Processing CreateTablet for tablet 4b626452d8bc4cc58823fe420b4dc549 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.936434 18902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b626452d8bc4cc58823fe420b4dc549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.936194 18978 tablet_service.cc:1511] Processing CreateTablet for tablet 4b626452d8bc4cc58823fe420b4dc549 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.937548 18978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b626452d8bc4cc58823fe420b4dc549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.938100 19052 tablet_service.cc:1511] Processing CreateTablet for tablet 4b626452d8bc4cc58823fe420b4dc549 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:30.939581 19052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b626452d8bc4cc58823fe420b4dc549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.940848 18903 tablet_service.cc:1511] Processing CreateTablet for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.942226 18903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.944729 19051 tablet_service.cc:1511] Processing CreateTablet for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.946169 19051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.947197 18977 tablet_service.cc:1511] Processing CreateTablet for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ba015f06da734f18990f9ee2841a4767]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:30.948480 18977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d1ceee59eb4f3fb31bc9a4010aa40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:30.955546 19102 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: Bootstrap starting.
I20260709 13:04:30.959616 19107 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: Bootstrap starting.
I20260709 13:04:30.962250 19102 tablet_bootstrap.cc:654] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.966620 19107 tablet_bootstrap.cc:654] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.971125 19102 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: No bootstrap required, opened a new log
I20260709 13:04:30.971845 19102 ts_tablet_manager.cc:1403] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: Time spent bootstrapping tablet: real 0.017s	user 0.005s	sys 0.009s
W20260709 13:04:30.972174 18859 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:30.975108 19102 raft_consensus.cc:359] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.976042 19102 raft_consensus.cc:385] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.976413 19102 raft_consensus.cc:740] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.977270 19102 consensus_queue.cc:260] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.978832 19107 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: No bootstrap required, opened a new log
I20260709 13:04:30.979486 19107 ts_tablet_manager.cc:1403] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:04:30.983052 19102 ts_tablet_manager.cc:1434] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:04:30.982789 19107 raft_consensus.cc:359] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.983820 19107 raft_consensus.cc:385] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:30.984191 19107 raft_consensus.cc:740] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Initialized, Role: FOLLOWER
I20260709 13:04:30.985420 19107 consensus_queue.cc:260] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:30.988116 19107 ts_tablet_manager.cc:1434] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:30.989555 19107 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: Bootstrap starting.
I20260709 13:04:30.994192 19103 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: Bootstrap starting.
I20260709 13:04:30.994880 19102 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Bootstrap starting.
I20260709 13:04:30.996773 19107 tablet_bootstrap.cc:654] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:30.997723 19106 raft_consensus.cc:493] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.998297 19106 raft_consensus.cc:515] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.002166 19102 tablet_bootstrap.cc:654] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:31.002179 19103 tablet_bootstrap.cc:654] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:31.009503 19106 leader_election.cc:290] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.009922 19103 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: No bootstrap required, opened a new log
I20260709 13:04:31.010514 19103 ts_tablet_manager.cc:1403] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:04:31.013950 19103 raft_consensus.cc:359] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.015400 19103 raft_consensus.cc:385] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.015918 19103 raft_consensus.cc:740] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.018141 19103 consensus_queue.cc:260] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.024056 19107 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: No bootstrap required, opened a new log
I20260709 13:04:31.024360 19103 ts_tablet_manager.cc:1434] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:04:31.024734 19107 ts_tablet_manager.cc:1403] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.000s
I20260709 13:04:31.025729 19103 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: Bootstrap starting.
I20260709 13:04:31.026244 19108 raft_consensus.cc:493] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.026795 19108 raft_consensus.cc:515] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.028030 19107 raft_consensus.cc:359] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.028797 19107 raft_consensus.cc:385] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.029088 19107 raft_consensus.cc:740] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.029928 19107 consensus_queue.cc:260] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.022300 19102 tablet_bootstrap.cc:492] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: No bootstrap required, opened a new log
I20260709 13:04:31.031256 19102 ts_tablet_manager.cc:1403] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.009s
I20260709 13:04:31.031102 19062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" is_pre_election: true
I20260709 13:04:31.032207 19062 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 0.
I20260709 13:04:31.034144 19103 tablet_bootstrap.cc:654] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:31.034720 19102 raft_consensus.cc:359] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.035445 19102 raft_consensus.cc:385] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.035733 19102 raft_consensus.cc:740] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.036157 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d" is_pre_election: true
I20260709 13:04:31.036938 19102 consensus_queue.cc:260] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.037345 18877 leader_election.cc:304] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 35a74035155c4a4cb1a0541d5966f32b; no voters: 
I20260709 13:04:31.038349 19115 raft_consensus.cc:2804] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:31.038703 19115 raft_consensus.cc:493] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.039004 19115 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.039636 19107 ts_tablet_manager.cc:1434] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:04:31.041684 18988 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 0.
I20260709 13:04:31.046710 19108 leader_election.cc:290] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.047855 19102 ts_tablet_manager.cc:1434] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:04:31.062729 19062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" is_pre_election: true
I20260709 13:04:31.063400 19103 tablet_bootstrap.cc:492] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: No bootstrap required, opened a new log
I20260709 13:04:31.063575 19062 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 0.
I20260709 13:04:31.063993 19103 ts_tablet_manager.cc:1403] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: Time spent bootstrapping tablet: real 0.038s	user 0.032s	sys 0.003s
I20260709 13:04:31.061270 19115 raft_consensus.cc:515] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.066499 18952 leader_election.cc:304] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:31.067471 19106 raft_consensus.cc:2804] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:31.067962 19106 raft_consensus.cc:493] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.067623 19103 raft_consensus.cc:359] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.068468 19106 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.068701 19103 raft_consensus.cc:385] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.069222 19103 raft_consensus.cc:740] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.069950 18913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a74035155c4a4cb1a0541d5966f32b" is_pre_election: true
I20260709 13:04:31.070133 19103 consensus_queue.cc:260] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.071834 18913 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 0.
I20260709 13:04:31.072969 19103 ts_tablet_manager.cc:1434] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:04:31.074048 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:31.074810 18988 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.075488 19115 leader_election.cc:290] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 election: Requested vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.076733 19062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:31.077461 19062 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.084336 18988 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 1.
I20260709 13:04:31.085004 19106 raft_consensus.cc:515] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.086158 19062 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 1.
I20260709 13:04:31.086366 18878 leader_election.cc:304] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a74035155c4a4cb1a0541d5966f32b, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:31.088076 19106 leader_election.cc:290] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 1 election: Requested vote from peers 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.088214 18913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
I20260709 13:04:31.088989 18913 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.088914 19115 raft_consensus.cc:2804] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:31.089620 19061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:31.089882 19115 raft_consensus.cc:697] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Becoming Leader. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:31.090358 19061 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.090909 19115 consensus_queue.cc:237] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.098006 19061 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 1.
I20260709 13:04:31.098062 18913 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 1.
I20260709 13:04:31.099414 18952 leader_election.cc:304] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:31.100370 19106 raft_consensus.cc:2804] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:31.101577 19106 raft_consensus.cc:697] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 LEADER]: Becoming Leader. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Running, Role: LEADER
I20260709 13:04:31.100530 18811 catalog_manager.cc:5697] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b reported cstate change: term changed from 0 to 1, leader changed from <none> to 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129). New cstate: current_term: 1 leader_uuid: "35a74035155c4a4cb1a0541d5966f32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.102626 19106 consensus_queue.cc:237] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.112434 18810 catalog_manager.cc:5697] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d reported cstate change: term changed from 0 to 1, leader changed from <none> to 74c46aafd0084285b93ed5785e30497d (127.7.117.130). New cstate: current_term: 1 leader_uuid: "74c46aafd0084285b93ed5785e30497d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.158134 19061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22c965f81d78414380429ba60d0362f8"
dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
mode: GRACEFUL
new_leader_uuid: "35a74035155c4a4cb1a0541d5966f32b"
 from {username='slave'} at 127.0.0.1:42652
I20260709 13:04:31.158859 19061 raft_consensus.cc:606] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 LEADER]: Received request to transfer leadership to TS 35a74035155c4a4cb1a0541d5966f32b
I20260709 13:04:31.172801 18913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8"
dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
mode: GRACEFUL
new_leader_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
 from {username='slave'} at 127.0.0.1:42368
I20260709 13:04:31.173478 18913 raft_consensus.cc:606] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Received request to transfer leadership to TS 2cc1e26cd1584388adcf5b07cf2778d8
I20260709 13:04:31.187228 18988 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549"
dest_uuid: "74c46aafd0084285b93ed5785e30497d"
mode: GRACEFUL
new_leader_uuid: "35a74035155c4a4cb1a0541d5966f32b"
 from {username='slave'} at 127.0.0.1:38890
I20260709 13:04:31.187731 18988 raft_consensus.cc:606] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 LEADER]: Received request to transfer leadership to TS 35a74035155c4a4cb1a0541d5966f32b
I20260709 13:04:31.197819 19115 consensus_queue.cc:1048] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:31.215176 19115 consensus_queue.cc:1048] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:31.231328 19108 raft_consensus.cc:993] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: : Instructing follower 2cc1e26cd1584388adcf5b07cf2778d8 to start an election
I20260709 13:04:31.231878 19115 raft_consensus.cc:1081] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Signalling peer 2cc1e26cd1584388adcf5b07cf2778d8 to start an election
I20260709 13:04:31.233613 19061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8"
dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
 from {username='slave'} at 127.0.0.1:42626
I20260709 13:04:31.234230 19061 raft_consensus.cc:493] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:31.234597 19061 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.241621 19061 raft_consensus.cc:515] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.243764 19061 leader_election.cc:290] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 2 election: Requested vote from peers 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963), 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477)
I20260709 13:04:31.244473 18913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
I20260709 13:04:31.245047 18913 raft_consensus.cc:3055] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:31.245352 18913 raft_consensus.cc:740] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:31.246045 18913 consensus_queue.cc:260] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.246901 18913 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.252185 18913 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 2.
I20260709 13:04:31.253720 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794bc6906476423296e3ea954ef14ea8" candidate_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:31.254098 19028 leader_election.cc:304] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 35a74035155c4a4cb1a0541d5966f32b; no voters: 
I20260709 13:04:31.254315 18988 raft_consensus.cc:3060] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.255803 19112 raft_consensus.cc:2804] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:31.256306 19112 raft_consensus.cc:697] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 LEADER]: Becoming Leader. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Running, Role: LEADER
I20260709 13:04:31.257148 19112 consensus_queue.cc:237] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.262605 18988 raft_consensus.cc:2468] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cc1e26cd1584388adcf5b07cf2778d8 in term 2.
I20260709 13:04:31.264667 18808 catalog_manager.cc:5697] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 reported cstate change: term changed from 1 to 2, leader changed from 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129) to 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.297524 19112 consensus_queue.cc:1048] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:31.309619 19118 consensus_queue.cc:1048] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:31.319232 19112 raft_consensus.cc:993] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: : Instructing follower 35a74035155c4a4cb1a0541d5966f32b to start an election
I20260709 13:04:31.319700 19118 raft_consensus.cc:1081] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 LEADER]: Signalling peer 35a74035155c4a4cb1a0541d5966f32b to start an election
I20260709 13:04:31.321359 18913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22c965f81d78414380429ba60d0362f8"
dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
 from {username='slave'} at 127.0.0.1:42346
I20260709 13:04:31.321976 18913 raft_consensus.cc:493] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:31.322350 18913 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.328495 18913 raft_consensus.cc:515] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.330526 18913 leader_election.cc:290] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 2 election: Requested vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.331509 19061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:31.332139 19061 raft_consensus.cc:3055] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:31.332466 19061 raft_consensus.cc:740] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2cc1e26cd1584388adcf5b07cf2778d8, State: Running, Role: LEADER
I20260709 13:04:31.333433 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c965f81d78414380429ba60d0362f8" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:31.333360 19061 consensus_queue.cc:260] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.333988 18988 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.334738 19061 raft_consensus.cc:3060] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.339607 18988 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 2.
I20260709 13:04:31.340008 19061 raft_consensus.cc:2468] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 2.
I20260709 13:04:31.340643 18878 leader_election.cc:304] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35a74035155c4a4cb1a0541d5966f32b, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:31.341476 19115 raft_consensus.cc:2804] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:31.341975 19115 raft_consensus.cc:697] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 LEADER]: Becoming Leader. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:31.342797 19115 consensus_queue.cc:237] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.349133 18811 catalog_manager.cc:5697] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b reported cstate change: term changed from 1 to 2, leader changed from 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131) to 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129). New cstate: current_term: 2 leader_uuid: "35a74035155c4a4cb1a0541d5966f32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.489735 19106 consensus_queue.cc:1048] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:31.496238 19134 raft_consensus.cc:993] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: : Instructing follower 35a74035155c4a4cb1a0541d5966f32b to start an election
I20260709 13:04:31.496908 19106 raft_consensus.cc:1081] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 LEADER]: Signalling peer 35a74035155c4a4cb1a0541d5966f32b to start an election
I20260709 13:04:31.498852 18912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549"
dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
 from {username='slave'} at 127.0.0.1:42352
I20260709 13:04:31.500480 18912 raft_consensus.cc:493] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:31.500874 18912 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.507542 19134 consensus_queue.cc:1048] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:31.511327 18912 raft_consensus.cc:515] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.516011 18912 leader_election.cc:290] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 2 election: Requested vote from peers 74c46aafd0084285b93ed5785e30497d (127.7.117.130:42477), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:31.517202 18988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74c46aafd0084285b93ed5785e30497d"
I20260709 13:04:31.517729 19061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b626452d8bc4cc58823fe420b4dc549" candidate_uuid: "35a74035155c4a4cb1a0541d5966f32b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:31.517891 18988 raft_consensus.cc:3055] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:31.518325 18988 raft_consensus.cc:740] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Running, Role: LEADER
I20260709 13:04:31.518330 19061 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.519290 18988 consensus_queue.cc:260] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.520695 18988 raft_consensus.cc:3060] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.526067 19061 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 2.
I20260709 13:04:31.527761 18877 leader_election.cc:304] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 35a74035155c4a4cb1a0541d5966f32b; no voters: 
I20260709 13:04:31.527635 18988 raft_consensus.cc:2468] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35a74035155c4a4cb1a0541d5966f32b in term 2.
I20260709 13:04:31.528954 19115 raft_consensus.cc:2804] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:31.529531 19115 raft_consensus.cc:697] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 2 LEADER]: Becoming Leader. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:31.530434 19115 consensus_queue.cc:237] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:31.539778 18811 catalog_manager.cc:5697] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b reported cstate change: term changed from 1 to 2, leader changed from 74c46aafd0084285b93ed5785e30497d (127.7.117.130) to 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129). New cstate: current_term: 2 leader_uuid: "35a74035155c4a4cb1a0541d5966f32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.578794 19115 consensus_queue.cc:1048] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:31.591486 19108 consensus_queue.cc:1048] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:31.671782 18988 raft_consensus.cc:1275] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Refusing update from remote peer 2cc1e26cd1584388adcf5b07cf2778d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.673278 19118 consensus_queue.cc:1048] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:31.681403 18912 raft_consensus.cc:1275] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Refusing update from remote peer 2cc1e26cd1584388adcf5b07cf2778d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.682963 19112 consensus_queue.cc:1048] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:31.693651  7638 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:04:31.695389  7638 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:04:31.695986  7638 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:04:31.708141 18912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e"
dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
mode: GRACEFUL
new_leader_uuid: "74c46aafd0084285b93ed5785e30497d"
 from {username='slave'} at 127.0.0.1:42376
I20260709 13:04:31.708702 18912 raft_consensus.cc:606] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Received request to transfer leadership to TS 74c46aafd0084285b93ed5785e30497d
I20260709 13:04:31.709810  7638 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:04:31.784590 19061 raft_consensus.cc:1275] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Refusing update from remote peer 35a74035155c4a4cb1a0541d5966f32b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.785864 19108 consensus_queue.cc:1048] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:31.795065 18988 raft_consensus.cc:1275] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Refusing update from remote peer 35a74035155c4a4cb1a0541d5966f32b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.796765 19139 consensus_queue.cc:1048] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:32.045293 18988 raft_consensus.cc:1275] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Refusing update from remote peer 35a74035155c4a4cb1a0541d5966f32b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:32.046639 19115 consensus_queue.cc:1048] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:32.054919 19061 raft_consensus.cc:1275] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Refusing update from remote peer 35a74035155c4a4cb1a0541d5966f32b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:32.056483 19115 consensus_queue.cc:1048] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:32.210024 19108 raft_consensus.cc:993] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: : Instructing follower 74c46aafd0084285b93ed5785e30497d to start an election
I20260709 13:04:32.210553 19115 raft_consensus.cc:1081] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Signalling peer 74c46aafd0084285b93ed5785e30497d to start an election
I20260709 13:04:32.212076 18988 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e"
dest_uuid: "74c46aafd0084285b93ed5785e30497d"
 from {username='slave'} at 127.0.0.1:38880
I20260709 13:04:32.212674 18988 raft_consensus.cc:493] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:32.213069 18988 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.218189 18988 raft_consensus.cc:515] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:32.222726 18988 leader_election.cc:290] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 2 election: Requested vote from peers 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129:45963), 2cc1e26cd1584388adcf5b07cf2778d8 (127.7.117.131:37675)
I20260709 13:04:32.222896 18912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35a74035155c4a4cb1a0541d5966f32b"
I20260709 13:04:32.223588 18912 raft_consensus.cc:3055] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:32.223738 19061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d1ceee59eb4f3fb31bc9a4010aa40e" candidate_uuid: "74c46aafd0084285b93ed5785e30497d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2cc1e26cd1584388adcf5b07cf2778d8"
I20260709 13:04:32.224011 18912 raft_consensus.cc:740] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35a74035155c4a4cb1a0541d5966f32b, State: Running, Role: LEADER
I20260709 13:04:32.224500 19061 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.225054 18912 consensus_queue.cc:260] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:32.226492 18912 raft_consensus.cc:3060] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.230003 19061 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 2.
I20260709 13:04:32.231344 18952 leader_election.cc:304] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc1e26cd1584388adcf5b07cf2778d8, 74c46aafd0084285b93ed5785e30497d; no voters: 
I20260709 13:04:32.232213 19134 raft_consensus.cc:2804] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:32.232255 18912 raft_consensus.cc:2468] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c46aafd0084285b93ed5785e30497d in term 2.
I20260709 13:04:32.233486 19134 raft_consensus.cc:697] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 LEADER]: Becoming Leader. State: Replica: 74c46aafd0084285b93ed5785e30497d, State: Running, Role: LEADER
I20260709 13:04:32.234272 19134 consensus_queue.cc:237] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } }
I20260709 13:04:32.241091 18810 catalog_manager.cc:5697] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d reported cstate change: term changed from 1 to 2, leader changed from 35a74035155c4a4cb1a0541d5966f32b (127.7.117.129) to 74c46aafd0084285b93ed5785e30497d (127.7.117.130). New cstate: current_term: 2 leader_uuid: "74c46aafd0084285b93ed5785e30497d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35a74035155c4a4cb1a0541d5966f32b" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 45963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c46aafd0084285b93ed5785e30497d" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 42477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc1e26cd1584388adcf5b07cf2778d8" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 37675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.334954  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 13:04:32.354351  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:32.355015  7638 tablet_replica.cc:333] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b: stopping tablet replica
I20260709 13:04:32.355659  7638 raft_consensus.cc:2243] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:32.356506  7638 raft_consensus.cc:2272] T 4b626452d8bc4cc58823fe420b4dc549 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.358812  7638 tablet_replica.cc:333] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b: stopping tablet replica
I20260709 13:04:32.359360  7638 raft_consensus.cc:2243] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.359746  7638 raft_consensus.cc:2272] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.361976  7638 tablet_replica.cc:333] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b: stopping tablet replica
I20260709 13:04:32.362481  7638 raft_consensus.cc:2243] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:32.363238  7638 raft_consensus.cc:2272] T 22c965f81d78414380429ba60d0362f8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.365370  7638 tablet_replica.cc:333] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b: stopping tablet replica
I20260709 13:04:32.365842  7638 raft_consensus.cc:2243] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.366259  7638 raft_consensus.cc:2272] T 794bc6906476423296e3ea954ef14ea8 P 35a74035155c4a4cb1a0541d5966f32b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.389691  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
I20260709 13:04:32.401896  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
I20260709 13:04:32.421468  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:32.422111  7638 tablet_replica.cc:333] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d: stopping tablet replica
I20260709 13:04:32.422670  7638 raft_consensus.cc:2243] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.423164  7638 raft_consensus.cc:2272] T 4b626452d8bc4cc58823fe420b4dc549 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.425294  7638 tablet_replica.cc:333] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d: stopping tablet replica
I20260709 13:04:32.425865  7638 raft_consensus.cc:2243] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.426298  7638 raft_consensus.cc:2272] T 794bc6906476423296e3ea954ef14ea8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.428467  7638 tablet_replica.cc:333] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: stopping tablet replica
I20260709 13:04:32.428985  7638 raft_consensus.cc:2243] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:32.429637  7638 pending_rounds.cc:70] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Trying to abort 1 pending ops.
I20260709 13:04:32.429816  7638 pending_rounds.cc:77] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305634907078426624 op_type: NO_OP noop_request { }
I20260709 13:04:32.430104  7638 raft_consensus.cc:2889] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:32.430414  7638 raft_consensus.cc:2272] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.432585  7638 tablet_replica.cc:333] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d: stopping tablet replica
I20260709 13:04:32.433079  7638 raft_consensus.cc:2243] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.433557  7638 raft_consensus.cc:2272] T 22c965f81d78414380429ba60d0362f8 P 74c46aafd0084285b93ed5785e30497d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.446295  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
I20260709 13:04:32.458626  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
I20260709 13:04:32.477025  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:32.477743  7638 tablet_replica.cc:333] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8: stopping tablet replica
I20260709 13:04:32.478346  7638 raft_consensus.cc:2243] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.478814  7638 raft_consensus.cc:2272] T 4b626452d8bc4cc58823fe420b4dc549 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.481091  7638 tablet_replica.cc:333] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8: stopping tablet replica
I20260709 13:04:32.481629  7638 raft_consensus.cc:2243] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:32.482350  7638 raft_consensus.cc:2272] T 794bc6906476423296e3ea954ef14ea8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.484449  7638 tablet_replica.cc:333] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8: stopping tablet replica
I20260709 13:04:32.484891  7638 raft_consensus.cc:2243] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.485247  7638 raft_consensus.cc:2272] T b5d1ceee59eb4f3fb31bc9a4010aa40e P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.487409  7638 tablet_replica.cc:333] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8: stopping tablet replica
I20260709 13:04:32.487818  7638 raft_consensus.cc:2243] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.488171  7638 raft_consensus.cc:2272] T 22c965f81d78414380429ba60d0362f8 P 2cc1e26cd1584388adcf5b07cf2778d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.510807  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 13:04:32.522976  7638 master.cc:562] Master@127.7.117.190:40945 shutting down...
I20260709 13:04:32.538250  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:32.538823  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.539155  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9eedf9c4f928455d903141210ff80f39: stopping tablet replica
I20260709 13:04:32.558118  7638 master.cc:584] Master@127.7.117.190:40945 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2746 ms)
[----------] 6 tests from LeaderRebalancerTest (336546 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:04:32.587862  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:39289
I20260709 13:04:32.589001  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.594175 19159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.594310 19160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.595800 19162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.596611  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:32.597505  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.597735  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.597898  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602272597883 us; error 0 us; skew 500 ppm
I20260709 13:04:32.598426  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.600701  7638 webserver.cc:533] Webserver started at http://127.7.117.190:44451/ using document root <none> and password file <none>
I20260709 13:04:32.601190  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.601424  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.601708  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.602785  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "d76cb895aa1246a18bbd4f78f4a10d6e"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.607352  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:04:32.610787 19167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.611786  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:04:32.612049  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "d76cb895aa1246a18bbd4f78f4a10d6e"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.612285  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.626209  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.627514  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.663834  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:39289
I20260709 13:04:32.663926 19218 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:39289 every 8 connection(s)
I20260709 13:04:32.667870 19219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.682327 19219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: Bootstrap starting.
I20260709 13:04:32.686964 19219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.691129 19219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: No bootstrap required, opened a new log
I20260709 13:04:32.693267 19219 raft_consensus.cc:359] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER }
I20260709 13:04:32.693797 19219 raft_consensus.cc:385] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.694041 19219 raft_consensus.cc:740] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76cb895aa1246a18bbd4f78f4a10d6e, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.694664 19219 consensus_queue.cc:260] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER }
I20260709 13:04:32.695118 19219 raft_consensus.cc:399] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:32.695358 19219 raft_consensus.cc:493] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:32.695626 19219 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:32.700654 19219 raft_consensus.cc:515] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER }
I20260709 13:04:32.701225 19219 leader_election.cc:304] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d76cb895aa1246a18bbd4f78f4a10d6e; no voters: 
I20260709 13:04:32.702503 19219 leader_election.cc:290] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:32.702862 19222 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:32.704314 19222 raft_consensus.cc:697] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 1 LEADER]: Becoming Leader. State: Replica: d76cb895aa1246a18bbd4f78f4a10d6e, State: Running, Role: LEADER
I20260709 13:04:32.704918 19222 consensus_queue.cc:237] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER }
I20260709 13:04:32.705502 19219 sys_catalog.cc:565] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:32.707742 19223 sys_catalog.cc:455] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER } }
I20260709 13:04:32.707741 19224 sys_catalog.cc:455] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d76cb895aa1246a18bbd4f78f4a10d6e. Latest consensus state: current_term: 1 leader_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76cb895aa1246a18bbd4f78f4a10d6e" member_type: VOTER } }
I20260709 13:04:32.708528 19223 sys_catalog.cc:458] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:32.708640 19224 sys_catalog.cc:458] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:32.713487 19227 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:32.721107 19227 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:32.722131  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:32.730255 19227 catalog_manager.cc:1383] Generated new cluster ID: a8ad1ee8f4b244ab8076bca616b447ed
I20260709 13:04:32.730568 19227 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:32.748515 19227 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:32.749902 19227 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:32.764834 19227 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: Generated new TSK 0
I20260709 13:04:32.765486 19227 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:32.788663  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.794661 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.795984 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.797816 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.797849  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:32.798986  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.799230  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.799417  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602272799393 us; error 0 us; skew 500 ppm
I20260709 13:04:32.799958  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.802345  7638 webserver.cc:533] Webserver started at http://127.7.117.129:33171/ using document root <none> and password file <none>
I20260709 13:04:32.802836  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.803045  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.803325  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.804457  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "ed29f8586d164987b3531a2710239e3d"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.809135  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:04:32.812510 19248 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.813505  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:32.813827  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "ed29f8586d164987b3531a2710239e3d"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.814210  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.827783  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.828958  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.830470  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.833134  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.833461  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:04:32.833688  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.833817  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.872762  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:44435
I20260709 13:04:32.872874 19310 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:44435 every 8 connection(s)
I20260709 13:04:32.878119  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.885663 19315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.886536 19316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.890245 19318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.891125  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:32.891626 19311 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39289
I20260709 13:04:32.892096 19311 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:32.893033 19311 heartbeater.cc:507] Master 127.7.117.190:39289 requested a full tablet report, sending...
I20260709 13:04:32.893146  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.893479  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.893673  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602272893658 us; error 0 us; skew 500 ppm
I20260709 13:04:32.894330  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.895704 19184 ts_manager.cc:194] Registered new tserver with Master: ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435)
I20260709 13:04:32.897104  7638 webserver.cc:533] Webserver started at http://127.7.117.130:39267/ using document root <none> and password file <none>
I20260709 13:04:32.897804  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.898039  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.898262 19184 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42080
I20260709 13:04:32.898361  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.899945  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "c4a3067ebfa947e18a871d0ac65c2786"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.904686  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:04:32.908046 19323 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.908845  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:32.909175  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "c4a3067ebfa947e18a871d0ac65c2786"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-ndzg"
I20260709 13:04:32.909531  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.954718  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.955791  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.957232  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.959492  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.959743  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.959987  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.960166  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.998596  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:38063
I20260709 13:04:32.998701 19385 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:38063 every 8 connection(s)
I20260709 13:04:33.003571  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:33.010568 19389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:33.012378 19390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:33.015391  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:33.016428 19386 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39289
W20260709 13:04:33.016606 19392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:33.016873 19386 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:33.017831  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:04:33.017930 19386 heartbeater.cc:507] Master 127.7.117.190:39289 requested a full tablet report, sending...
W20260709 13:04:33.018154  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:33.018397  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602273018378 us; error 0 us; skew 500 ppm
I20260709 13:04:33.019156  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:33.020380 19184 ts_manager.cc:194] Registered new tserver with Master: c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:33.022006 19184 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42088
I20260709 13:04:33.022136  7638 webserver.cc:533] Webserver started at http://127.7.117.131:43039/ using document root <none> and password file <none>
I20260709 13:04:33.022773  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:33.023018  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:33.023327  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:33.024407  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "f825e4d993594c96a14fcb1a1c742ef2"
format_stamp: "Formatted at 2026-07-09 13:04:33 on dist-test-slave-ndzg"
I20260709 13:04:33.028714  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:04:33.031931 19397 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:33.032845  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:33.033098  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "f825e4d993594c96a14fcb1a1c742ef2"
format_stamp: "Formatted at 2026-07-09 13:04:33 on dist-test-slave-ndzg"
I20260709 13:04:33.033406  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:33.055902  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:33.056936  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:33.058348  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:33.060616  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:33.060793  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:33.061095  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:33.061235  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:33.099323  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:38493
I20260709 13:04:33.099447 19459 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:38493 every 8 connection(s)
I20260709 13:04:33.113178 19460 heartbeater.cc:344] Connected to a master server at 127.7.117.190:39289
I20260709 13:04:33.113758 19460 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:33.114588 19460 heartbeater.cc:507] Master 127.7.117.190:39289 requested a full tablet report, sending...
I20260709 13:04:33.116631 19184 ts_manager.cc:194] Registered new tserver with Master: f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.117190  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014797067s
I20260709 13:04:33.118440 19184 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42092
I20260709 13:04:33.142463 19184 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42100:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:33.144976 19184 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:33.225250 19351 tablet_service.cc:1511] Processing CreateTablet for tablet e96e9a71181b4523b82e78bc0e093f5e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:33.225667 19350 tablet_service.cc:1511] Processing CreateTablet for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:33.226740 19351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96e9a71181b4523b82e78bc0e093f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.227401 19350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.227167 19349 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc52ac04e444c4b343c7794b8db3bb (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:33.228562 19349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc52ac04e444c4b343c7794b8db3bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.230396 19348 tablet_service.cc:1511] Processing CreateTablet for tablet 321f8b0faf5b4e4a88fab2da1e36f384 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:33.231841 19348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321f8b0faf5b4e4a88fab2da1e36f384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.234371 19347 tablet_service.cc:1511] Processing CreateTablet for tablet dd76380481434977ab62d183db59a9d2 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:33.235867 19347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76380481434977ab62d183db59a9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.238371 19346 tablet_service.cc:1511] Processing CreateTablet for tablet 153e8097ed504596872f548f628f6d71 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:33.241097 19346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153e8097ed504596872f548f628f6d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.240904 19344 tablet_service.cc:1511] Processing CreateTablet for tablet 50b5b57968404d72bb79778c9beb66fc (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:33.242304 19344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b5b57968404d72bb79778c9beb66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.230396 19345 tablet_service.cc:1511] Processing CreateTablet for tablet 48d6ee85a0fc449db61e9360433be5d3 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:33.243786 19345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d6ee85a0fc449db61e9360433be5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.253137 19276 tablet_service.cc:1511] Processing CreateTablet for tablet e96e9a71181b4523b82e78bc0e093f5e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:33.253137 19343 tablet_service.cc:1511] Processing CreateTablet for tablet 86e4e8a1c6994730bc90ae084a99faad (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:33.254854 19276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96e9a71181b4523b82e78bc0e093f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.254874 19343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e4e8a1c6994730bc90ae084a99faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.267750 19275 tablet_service.cc:1511] Processing CreateTablet for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:33.270562 19275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.279775 19274 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc52ac04e444c4b343c7794b8db3bb (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:33.281960 19274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc52ac04e444c4b343c7794b8db3bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.283862 19425 tablet_service.cc:1511] Processing CreateTablet for tablet e96e9a71181b4523b82e78bc0e093f5e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:33.285025 19425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96e9a71181b4523b82e78bc0e093f5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.291775 19273 tablet_service.cc:1511] Processing CreateTablet for tablet 321f8b0faf5b4e4a88fab2da1e36f384 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:33.293871 19273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321f8b0faf5b4e4a88fab2da1e36f384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.295009 19424 tablet_service.cc:1511] Processing CreateTablet for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:33.296414 19424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faf2a5f8d2a4600b8539dce8cbf9f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.299718 19272 tablet_service.cc:1511] Processing CreateTablet for tablet dd76380481434977ab62d183db59a9d2 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:33.300800 19423 tablet_service.cc:1511] Processing CreateTablet for tablet cbcc52ac04e444c4b343c7794b8db3bb (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:33.301725 19272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76380481434977ab62d183db59a9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.302337 19423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbcc52ac04e444c4b343c7794b8db3bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.304970 19422 tablet_service.cc:1511] Processing CreateTablet for tablet 321f8b0faf5b4e4a88fab2da1e36f384 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:33.306438 19422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321f8b0faf5b4e4a88fab2da1e36f384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.315219 19271 tablet_service.cc:1511] Processing CreateTablet for tablet 153e8097ed504596872f548f628f6d71 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:33.316694 19271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153e8097ed504596872f548f628f6d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.322417 19270 tablet_service.cc:1511] Processing CreateTablet for tablet 48d6ee85a0fc449db61e9360433be5d3 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:33.323140 19421 tablet_service.cc:1511] Processing CreateTablet for tablet dd76380481434977ab62d183db59a9d2 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:33.323909 19270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d6ee85a0fc449db61e9360433be5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.324533 19421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76380481434977ab62d183db59a9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.326387 19419 tablet_service.cc:1511] Processing CreateTablet for tablet 48d6ee85a0fc449db61e9360433be5d3 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:33.327807 19419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d6ee85a0fc449db61e9360433be5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.330443 19418 tablet_service.cc:1511] Processing CreateTablet for tablet 50b5b57968404d72bb79778c9beb66fc (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:33.331831 19418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b5b57968404d72bb79778c9beb66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.334511 19417 tablet_service.cc:1511] Processing CreateTablet for tablet 86e4e8a1c6994730bc90ae084a99faad (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:33.335898 19417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e4e8a1c6994730bc90ae084a99faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.336529 19269 tablet_service.cc:1511] Processing CreateTablet for tablet 50b5b57968404d72bb79778c9beb66fc (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:33.337895 19269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b5b57968404d72bb79778c9beb66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.337926 19420 tablet_service.cc:1511] Processing CreateTablet for tablet 153e8097ed504596872f548f628f6d71 (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:33.339421 19420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153e8097ed504596872f548f628f6d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.339876 19268 tablet_service.cc:1511] Processing CreateTablet for tablet 86e4e8a1c6994730bc90ae084a99faad (DEFAULT_TABLE table=test-workload [id=9e09cee5814f4116b22cd9c3e7f458f5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:33.341594 19268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e4e8a1c6994730bc90ae084a99faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.359565 19475 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.378607 19476 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.384775 19476 tablet_bootstrap.cc:654] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:33.386565 19170 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.117.130:38063, user_credentials={real_user=slave}} blocked reactor thread for 49545.3us
I20260709 13:04:33.451538 19476 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.452364 19476 ts_tablet_manager.cc:1403] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.017s
I20260709 13:04:33.456399 19476 raft_consensus.cc:359] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.457454 19476 raft_consensus.cc:385] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.457851 19476 raft_consensus.cc:740] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.458909 19476 consensus_queue.cc:260] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.464906 19475 tablet_bootstrap.cc:654] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.465585 19311 heartbeater.cc:499] Master 127.7.117.190:39289 was elected leader, sending a full tablet report...
I20260709 13:04:33.466180 19476 ts_tablet_manager.cc:1434] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.009s
I20260709 13:04:33.465893 19478 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.467226 19476 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.474241 19476 tablet_bootstrap.cc:654] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.474332 19478 tablet_bootstrap.cc:654] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.478420 19475 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.478978 19475 ts_tablet_manager.cc:1403] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.000s
I20260709 13:04:33.481997 19475 raft_consensus.cc:359] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.482781 19475 raft_consensus.cc:385] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.483073 19475 raft_consensus.cc:740] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.483883 19475 consensus_queue.cc:260] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.492373 19478 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.492789 19386 heartbeater.cc:499] Master 127.7.117.190:39289 was elected leader, sending a full tablet report...
I20260709 13:04:33.493012 19478 ts_tablet_manager.cc:1403] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.004s
I20260709 13:04:33.493079 19475 ts_tablet_manager.cc:1434] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.012s
I20260709 13:04:33.494436 19475 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.494874 19476 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.495474 19476 ts_tablet_manager.cc:1403] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.006s
I20260709 13:04:33.496315 19478 raft_consensus.cc:359] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.497143 19478 raft_consensus.cc:385] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.497462 19478 raft_consensus.cc:740] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.498402 19478 consensus_queue.cc:260] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.498564 19476 raft_consensus.cc:359] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.499465 19476 raft_consensus.cc:385] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.499878 19476 raft_consensus.cc:740] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.500519 19475 tablet_bootstrap.cc:654] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.501170 19476 consensus_queue.cc:260] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.504945 19476 ts_tablet_manager.cc:1434] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:04:33.507735 19483 raft_consensus.cc:493] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.508476 19483 raft_consensus.cc:515] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.512770 19476 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.519412 19483 leader_election.cc:290] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493), ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435)
I20260709 13:04:33.517117 19478 ts_tablet_manager.cc:1434] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 13:04:33.520655 19460 heartbeater.cc:499] Master 127.7.117.190:39289 was elected leader, sending a full tablet report...
I20260709 13:04:33.529172 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
W20260709 13:04:33.530964 19325 leader_election.cc:343] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.532186 19478 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.537106 19475 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.537781 19475 ts_tablet_manager.cc:1403] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.005s
I20260709 13:04:33.540813 19475 raft_consensus.cc:359] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.541594 19475 raft_consensus.cc:385] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.541908 19475 raft_consensus.cc:740] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.542410 19478 tablet_bootstrap.cc:654] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.542641 19475 consensus_queue.cc:260] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.545042 19475 ts_tablet_manager.cc:1434] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:04:33.546083 19475 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.551427 19476 tablet_bootstrap.cc:654] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.552227 19475 tablet_bootstrap.cc:654] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.559227 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:33.560035 19286 raft_consensus.cc:2468] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:33.561509 19326 leader_election.cc:304] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:33.563292 19483 raft_consensus.cc:2804] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.563643 19483 raft_consensus.cc:493] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.563941 19483 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.554019 19478 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.569571 19478 ts_tablet_manager.cc:1403] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.007s
I20260709 13:04:33.572073 19483 raft_consensus.cc:515] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.572443 19478 raft_consensus.cc:359] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.573339 19478 raft_consensus.cc:385] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.573717 19478 raft_consensus.cc:740] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.574571 19478 consensus_queue.cc:260] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.575206 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
W20260709 13:04:33.576668 19325 leader_election.cc:343] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.576900 19478 ts_tablet_manager.cc:1434] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:04:33.578078 19478 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.578859 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:33.579533 19286 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.584226 19478 tablet_bootstrap.cc:654] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.588136 19286 raft_consensus.cc:2468] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:33.589975 19326 leader_election.cc:304] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:33.595311 19483 leader_election.cc:290] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493), ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435)
I20260709 13:04:33.596271 19483 raft_consensus.cc:2804] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.596860 19483 raft_consensus.cc:697] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:33.598055 19483 consensus_queue.cc:237] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.610831 19476 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.611465 19476 ts_tablet_manager.cc:1403] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.099s	user 0.039s	sys 0.037s
I20260709 13:04:33.614236 19476 raft_consensus.cc:359] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.615108 19476 raft_consensus.cc:385] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.615500 19476 raft_consensus.cc:740] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.616442 19476 consensus_queue.cc:260] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.617669 19475 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.618315 19475 ts_tablet_manager.cc:1403] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.028s
I20260709 13:04:33.621448 19475 raft_consensus.cc:359] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.623616 19475 raft_consensus.cc:385] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.624002 19475 raft_consensus.cc:740] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.624862 19475 consensus_queue.cc:260] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.625257 19478 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.626076 19476 ts_tablet_manager.cc:1434] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 13:04:33.626179 19478 ts_tablet_manager.cc:1403] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.048s	user 0.030s	sys 0.016s
I20260709 13:04:33.629390 19475 ts_tablet_manager.cc:1434] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:33.630640 19475 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.630636 19478 raft_consensus.cc:359] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.631995 19478 raft_consensus.cc:385] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.632421 19478 raft_consensus.cc:740] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.633693 19478 consensus_queue.cc:260] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.634261 19476 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.639547 19475 tablet_bootstrap.cc:654] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.642026 19476 tablet_bootstrap.cc:654] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.646591 19478 ts_tablet_manager.cc:1434] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.005s
I20260709 13:04:33.651902 19478 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.652904 19184 catalog_manager.cc:5697] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.655042 19475 tablet_bootstrap.cc:492] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.655640 19475 ts_tablet_manager.cc:1403] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.002s
I20260709 13:04:33.658607 19475 raft_consensus.cc:359] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.659330 19475 raft_consensus.cc:385] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.659462 19476 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.659667 19475 raft_consensus.cc:740] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.660418 19476 ts_tablet_manager.cc:1403] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 13:04:33.660656 19475 consensus_queue.cc:260] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.663053 19475 ts_tablet_manager.cc:1434] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:33.663596 19476 raft_consensus.cc:359] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.664179 19475 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.664500 19476 raft_consensus.cc:385] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.664827 19476 raft_consensus.cc:740] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.665752 19476 consensus_queue.cc:260] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.669628 19479 raft_consensus.cc:493] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.670125 19479 raft_consensus.cc:515] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.671262 19478 tablet_bootstrap.cc:654] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.671420 19475 tablet_bootstrap.cc:654] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.677620 19479 leader_election.cc:290] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.678293 19479 raft_consensus.cc:493] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.678740 19479 raft_consensus.cc:515] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.683593 19479 leader_election.cc:290] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.688802 19476 ts_tablet_manager.cc:1434] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:04:33.693198 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc52ac04e444c4b343c7794b8db3bb" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:33.694406 19361 raft_consensus.cc:2468] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:33.696346 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:33.699040 19251 leader_election.cc:304] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
W20260709 13:04:33.700361 19251 leader_election.cc:343] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.702184 19497 raft_consensus.cc:2804] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.703042 19497 raft_consensus.cc:493] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.703578 19497 raft_consensus.cc:3060] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.703899 19476 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.707146 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc52ac04e444c4b343c7794b8db3bb" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.707957 19435 raft_consensus.cc:2468] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:33.709605 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.712087 19484 raft_consensus.cc:493] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.712852 19250 leader_election.cc:343] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.712594 19484 raft_consensus.cc:515] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.713402 19250 leader_election.cc:304] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed29f8586d164987b3531a2710239e3d; no voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:33.717432 19479 raft_consensus.cc:2749] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:33.718906 19232 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:33.718616 19497 raft_consensus.cc:515] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.719813 19484 leader_election.cc:290] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:33.722172 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc52ac04e444c4b343c7794b8db3bb" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:33.722901 19360 raft_consensus.cc:3060] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.729621 19476 tablet_bootstrap.cc:654] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.733616 19475 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.734362 19475 ts_tablet_manager.cc:1403] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.070s	user 0.027s	sys 0.036s
I20260709 13:04:33.734877 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbcc52ac04e444c4b343c7794b8db3bb" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:33.735598 19434 raft_consensus.cc:3060] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.736377 19497 leader_election.cc:290] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.738696 19475 raft_consensus.cc:359] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.739367 19360 raft_consensus.cc:2468] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:33.739575 19475 raft_consensus.cc:385] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.739948 19475 raft_consensus.cc:740] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.740814 19251 leader_election.cc:304] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:33.740864 19475 consensus_queue.cc:260] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.741986 19497 raft_consensus.cc:2804] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.743211 19497 raft_consensus.cc:697] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:33.743487 19475 ts_tablet_manager.cc:1434] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:04:33.744616 19475 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.749058 19483 raft_consensus.cc:493] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.753795 19483 raft_consensus.cc:515] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.744087 19497 consensus_queue.cc:237] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.756879 19434 raft_consensus.cc:2468] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:33.758726 19478 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.759441 19478 ts_tablet_manager.cc:1403] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.108s	user 0.026s	sys 0.022s
I20260709 13:04:33.768477 19483 leader_election.cc:290] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.759584 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.769371 19435 raft_consensus.cc:2468] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:33.770223 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e4e8a1c6994730bc90ae084a99faad" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:33.770766 19325 leader_election.cc:304] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:33.771072 19476 tablet_bootstrap.cc:492] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.770960 19360 raft_consensus.cc:2468] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:33.760375 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:33.771816 19476 ts_tablet_manager.cc:1403] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.006s
I20260709 13:04:33.761020 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e4e8a1c6994730bc90ae084a99faad" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:33.772691 19400 leader_election.cc:304] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
W20260709 13:04:33.773039 19326 leader_election.cc:343] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.773748 19484 raft_consensus.cc:2804] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.774127 19484 raft_consensus.cc:493] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.766146 19475 tablet_bootstrap.cc:654] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.774458 19484 raft_consensus.cc:3060] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.778383 19476 raft_consensus.cc:359] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.779312 19476 raft_consensus.cc:385] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.779592 19476 raft_consensus.cc:740] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.780937 19476 consensus_queue.cc:260] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.774442 19483 raft_consensus.cc:2804] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.782101 19483 raft_consensus.cc:493] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.782655 19483 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.783318 19476 ts_tablet_manager.cc:1434] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:33.783502 19182 catalog_manager.cc:5697] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 1 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.784499 19476 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.791792 19476 tablet_bootstrap.cc:654] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:33.792655 19400 leader_election.cc:343] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.794654 19484 raft_consensus.cc:515] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.794723 19478 raft_consensus.cc:359] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.795709 19478 raft_consensus.cc:385] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.795919 19475 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.796085 19478 raft_consensus.cc:740] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.796564 19475 ts_tablet_manager.cc:1403] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.005s
I20260709 13:04:33.797129 19478 consensus_queue.cc:260] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.798151 19479 raft_consensus.cc:493] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.798172 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e4e8a1c6994730bc90ae084a99faad" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:33.798816 19479 raft_consensus.cc:515] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.799183 19285 raft_consensus.cc:3060] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.799822 19475 raft_consensus.cc:359] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.800612 19475 raft_consensus.cc:385] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.800905 19475 raft_consensus.cc:740] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.801707 19475 consensus_queue.cc:260] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.802567 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:33.803993 19475 ts_tablet_manager.cc:1434] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
W20260709 13:04:33.804781 19251 leader_election.cc:343] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.805444 19479 leader_election.cc:290] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.805732 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.805083 19475 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
W20260709 13:04:33.808130 19250 leader_election.cc:343] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.808737 19250 leader_election.cc:304] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed29f8586d164987b3531a2710239e3d; no voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:33.814224 19497 raft_consensus.cc:2749] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.816679 19483 raft_consensus.cc:515] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.818933 19483 leader_election.cc:290] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.819208 19475 tablet_bootstrap.cc:654] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.819648 19484 leader_election.cc:290] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:33.820763 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e4e8a1c6994730bc90ae084a99faad" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:33.821524 19360 raft_consensus.cc:3060] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.822400 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:33.823216 19435 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.828657 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
W20260709 13:04:33.830832 19326 leader_election.cc:343] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.832762 19360 raft_consensus.cc:2468] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:33.834602 19400 leader_election.cc:304] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:33.835829 19484 raft_consensus.cc:2804] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.836347 19484 raft_consensus.cc:697] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:33.837145 19484 consensus_queue.cc:237] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.848253 19435 raft_consensus.cc:2468] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:33.850006 19325 leader_election.cc:304] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: ed29f8586d164987b3531a2710239e3d
I20260709 13:04:33.851045 19483 raft_consensus.cc:2804] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.818310 19285 raft_consensus.cc:2468] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:33.851564 19483 raft_consensus.cc:697] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:33.852505 19483 consensus_queue.cc:237] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.870189 19478 ts_tablet_manager.cc:1434] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.031s
I20260709 13:04:33.870177 19476 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.871064 19476 ts_tablet_manager.cc:1403] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.087s	user 0.033s	sys 0.025s
I20260709 13:04:33.871544 19478 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.872396 19182 catalog_manager.cc:5697] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.874143 19476 raft_consensus.cc:359] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.873696 19184 catalog_manager.cc:5697] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.874856 19476 raft_consensus.cc:385] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.875203 19476 raft_consensus.cc:740] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.876112 19476 consensus_queue.cc:260] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.878399 19475 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.879014 19475 ts_tablet_manager.cc:1403] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.020s
I20260709 13:04:33.879096 19478 tablet_bootstrap.cc:654] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.881043 19476 ts_tablet_manager.cc:1434] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 13:04:33.882206 19476 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.894575 19475 raft_consensus.cc:359] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.895519 19475 raft_consensus.cc:385] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.895871 19475 raft_consensus.cc:740] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.897272 19476 tablet_bootstrap.cc:654] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.896811 19475 consensus_queue.cc:260] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.901744 19475 ts_tablet_manager.cc:1434] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:04:33.902938 19475 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.909812 19475 tablet_bootstrap.cc:654] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.913638 19478 tablet_bootstrap.cc:492] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.914207 19478 ts_tablet_manager.cc:1403] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.019s
I20260709 13:04:33.915194 19475 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.915719 19475 ts_tablet_manager.cc:1403] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:04:33.916550 19478 raft_consensus.cc:359] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.917128 19478 raft_consensus.cc:385] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.917503 19478 raft_consensus.cc:740] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.917883 19475 raft_consensus.cc:359] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.918428 19475 raft_consensus.cc:385] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.918728 19475 raft_consensus.cc:740] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.918239 19478 consensus_queue.cc:260] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.919354 19475 consensus_queue.cc:260] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.920822 19478 ts_tablet_manager.cc:1434] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:33.922019 19478 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.923417 19476 tablet_bootstrap.cc:492] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.923947 19476 ts_tablet_manager.cc:1403] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.042s	user 0.002s	sys 0.011s
I20260709 13:04:33.927000 19476 raft_consensus.cc:359] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.927704 19476 raft_consensus.cc:385] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.927920 19476 raft_consensus.cc:740] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.928423 19476 consensus_queue.cc:260] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.929404 19478 tablet_bootstrap.cc:654] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.930548 19476 ts_tablet_manager.cc:1434] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:04:33.931640 19476 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.921195 19475 ts_tablet_manager.cc:1434] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 13:04:33.934181 19475 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:33.939942 19475 tablet_bootstrap.cc:654] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.944873 19478 tablet_bootstrap.cc:492] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.945443 19478 ts_tablet_manager.cc:1403] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.004s
I20260709 13:04:33.947253 19475 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:33.947592 19476 tablet_bootstrap.cc:654] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.947726 19475 ts_tablet_manager.cc:1403] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 13:04:33.948489 19478 raft_consensus.cc:359] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.949312 19478 raft_consensus.cc:385] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.949631 19478 raft_consensus.cc:740] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.950400 19475 raft_consensus.cc:359] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.950418 19478 consensus_queue.cc:260] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:33.951167 19475 raft_consensus.cc:385] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.951561 19475 raft_consensus.cc:740] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.952234 19475 consensus_queue.cc:260] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.953101 19478 ts_tablet_manager.cc:1434] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:33.953202 19476 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.953980 19476 ts_tablet_manager.cc:1403] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.000s
I20260709 13:04:33.954196 19478 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.954471 19475 ts_tablet_manager.cc:1434] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:04:33.956911 19476 raft_consensus.cc:359] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.957715 19476 raft_consensus.cc:385] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.958051 19476 raft_consensus.cc:740] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.958835 19476 consensus_queue.cc:260] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.961485 19478 tablet_bootstrap.cc:654] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.963781 19476 ts_tablet_manager.cc:1434] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:04:33.965270 19476 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:33.973084 19476 tablet_bootstrap.cc:654] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.975796 19478 tablet_bootstrap.cc:492] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:33.976333 19478 ts_tablet_manager.cc:1403] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.016s
I20260709 13:04:33.978593 19476 tablet_bootstrap.cc:492] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:33.979091 19476 ts_tablet_manager.cc:1403] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:04:33.979068 19478 raft_consensus.cc:359] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.979864 19478 raft_consensus.cc:385] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.980103 19478 raft_consensus.cc:740] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.980700 19478 consensus_queue.cc:260] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.981793 19476 raft_consensus.cc:359] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.982388 19497 raft_consensus.cc:493] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.982445 19476 raft_consensus.cc:385] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.982756 19478 ts_tablet_manager.cc:1434] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:33.982988 19476 raft_consensus.cc:740] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.982990 19497 raft_consensus.cc:515] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.984026 19478 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:33.984061 19476 consensus_queue.cc:260] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:33.986795 19476 ts_tablet_manager.cc:1434] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:33.986616 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:33.988019 19360 raft_consensus.cc:2468] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:33.988837 19483 raft_consensus.cc:493] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.989146 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.989498 19483 raft_consensus.cc:515] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
W20260709 13:04:33.991247 19250 leader_election.cc:343] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.991779 19483 leader_election.cc:290] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:33.991720 19478 tablet_bootstrap.cc:654] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.992013 19251 leader_election.cc:304] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:33.992972 19479 raft_consensus.cc:2804] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.993371 19479 raft_consensus.cc:493] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.993688 19479 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.994109 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d6ee85a0fc449db61e9360433be5d3" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:33.994802 19285 raft_consensus.cc:2468] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:33.995831 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d6ee85a0fc449db61e9360433be5d3" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:33.996497 19435 raft_consensus.cc:2468] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:33.997809 19325 leader_election.cc:304] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:33.998694 19483 raft_consensus.cc:2804] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.999081 19483 raft_consensus.cc:493] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.999455 19483 raft_consensus.cc:3060] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.002399 19479 raft_consensus.cc:515] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.003268 19478 tablet_bootstrap.cc:492] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:34.003682 19478 ts_tablet_manager.cc:1403] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:04:34.004832 19479 leader_election.cc:290] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.005745 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:34.005918 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.006383 19360 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.006242 19478 raft_consensus.cc:359] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.007055 19478 raft_consensus.cc:385] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:34.007094 19250 leader_election.cc:343] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.007465 19478 raft_consensus.cc:740] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.008208 19478 consensus_queue.cc:260] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.010345 19478 ts_tablet_manager.cc:1434] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:04:34.011499 19478 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:34.011469 19497 leader_election.cc:290] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.018534 19478 tablet_bootstrap.cc:654] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.019459 19483 raft_consensus.cc:515] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.022501 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d6ee85a0fc449db61e9360433be5d3" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:34.023115 19285 raft_consensus.cc:3060] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.024230 19478 tablet_bootstrap.cc:492] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:34.024569 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d6ee85a0fc449db61e9360433be5d3" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.024778 19478 ts_tablet_manager.cc:1403] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:04:34.025161 19360 raft_consensus.cc:2468] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:34.025177 19435 raft_consensus.cc:3060] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.026614 19251 leader_election.cc:304] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:34.027683 19478 raft_consensus.cc:359] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.027740 19497 raft_consensus.cc:2804] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.028417 19478 raft_consensus.cc:385] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.028915 19478 raft_consensus.cc:740] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.028975 19497 raft_consensus.cc:697] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:34.029815 19478 consensus_queue.cc:260] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.029979 19497 consensus_queue.cc:237] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.031951 19478 ts_tablet_manager.cc:1434] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:34.034844 19285 raft_consensus.cc:2468] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.036012 19326 leader_election.cc:304] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:34.037175 19491 raft_consensus.cc:2804] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.037705 19491 raft_consensus.cc:697] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.038005 19483 leader_election.cc:290] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.038543 19491 consensus_queue.cc:237] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.041494 19483 consensus_queue.cc:1048] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.041487 19435 raft_consensus.cc:2468] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.047927 19181 catalog_manager.cc:5697] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 1 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.052294 19183 catalog_manager.cc:5697] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.060472 19435 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.068930 19520 consensus_queue.cc:1048] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.111907 19491 raft_consensus.cc:493] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.112319 19491 raft_consensus.cc:515] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.114063 19491 leader_election.cc:290] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.115072 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:34.115072 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:34.116089 19435 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.116165 19285 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.117182 19325 leader_election.cc:304] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:34.117899 19491 raft_consensus.cc:2804] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.118202 19491 raft_consensus.cc:493] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.118508 19491 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.123780 19491 raft_consensus.cc:515] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.125424 19491 leader_election.cc:290] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.126235 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:34.126374 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.126780 19285 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.126995 19435 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.131731 19285 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.131745 19435 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.132800 19326 leader_election.cc:304] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:34.133622 19491 raft_consensus.cc:2804] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.134087 19491 raft_consensus.cc:697] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.135150 19491 consensus_queue.cc:237] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.139006 19497 raft_consensus.cc:493] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.139554 19497 raft_consensus.cc:515] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.141580 19497 leader_election.cc:290] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.142280 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:34.143139 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:34.143957 19434 raft_consensus.cc:2393] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed29f8586d164987b3531a2710239e3d in current term 1: Already voted for candidate c4a3067ebfa947e18a871d0ac65c2786 in this term.
I20260709 13:04:34.143807 19183 catalog_manager.cc:5697] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.144397 19497 consensus_queue.cc:1048] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.144996 19250 leader_election.cc:304] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed29f8586d164987b3531a2710239e3d; no voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:34.145853 19497 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.151358 19491 raft_consensus.cc:493] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.151978 19497 raft_consensus.cc:2749] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.152088 19491 raft_consensus.cc:515] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.155835 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76380481434977ab62d183db59a9d2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:34.156316 19491 leader_election.cc:290] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.156526 19285 raft_consensus.cc:2468] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.157207 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76380481434977ab62d183db59a9d2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:34.157820 19326 leader_election.cc:304] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:34.158028 19435 raft_consensus.cc:2468] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.158738 19491 raft_consensus.cc:2804] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.159355 19491 raft_consensus.cc:493] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.159817 19491 raft_consensus.cc:3060] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.167511 19491 raft_consensus.cc:515] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.170135 19497 consensus_queue.cc:1048] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.170421 19491 leader_election.cc:290] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.172555 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76380481434977ab62d183db59a9d2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.173213 19434 raft_consensus.cc:3060] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.177332 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76380481434977ab62d183db59a9d2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:34.177996 19285 raft_consensus.cc:3060] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.184659 19285 raft_consensus.cc:2468] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.184729 19434 raft_consensus.cc:2468] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.185720 19326 leader_election.cc:304] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:34.186431 19491 raft_consensus.cc:2804] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.186776 19491 raft_consensus.cc:697] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.187397 19491 consensus_queue.cc:237] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.194422 19183 catalog_manager.cc:5697] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.252249 19491 consensus_queue.cc:1048] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.266702 19483 consensus_queue.cc:1048] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.309913 19483 raft_consensus.cc:493] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.310427 19483 raft_consensus.cc:515] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.312288 19483 leader_election.cc:290] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.313118 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:34.313351 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:34.313925 19434 raft_consensus.cc:2468] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.313925 19285 raft_consensus.cc:2468] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:34.315090 19325 leader_election.cc:304] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:34.315887 19483 raft_consensus.cc:2804] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.316406 19483 raft_consensus.cc:493] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.316725 19483 raft_consensus.cc:3060] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.317246 19504 consensus_queue.cc:1048] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.323720 19483 raft_consensus.cc:515] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.327133 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:34.327746 19286 raft_consensus.cc:3060] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.331338 19483 leader_election.cc:290] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.332050 19535 consensus_queue.cc:1048] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.332369 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153e8097ed504596872f548f628f6d71" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.332962 19434 raft_consensus.cc:3060] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.335355 19286 raft_consensus.cc:2468] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.336544 19326 leader_election.cc:304] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:34.337584 19520 raft_consensus.cc:2804] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.338081 19520 raft_consensus.cc:697] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.338984 19520 consensus_queue.cc:237] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.342077 19434 raft_consensus.cc:2468] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:34.347453 19183 catalog_manager.cc:5697] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.451534 19520 consensus_queue.cc:1048] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.462744 19491 consensus_queue.cc:1048] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.478871 19535 raft_consensus.cc:493] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.479287 19535 raft_consensus.cc:515] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.480837 19535 leader_election.cc:290] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:34.481590 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:34.481832 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:34.482599 19360 raft_consensus.cc:2393] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f825e4d993594c96a14fcb1a1c742ef2 in current term 1: Already voted for candidate ed29f8586d164987b3531a2710239e3d in this term.
I20260709 13:04:34.483512 19400 leader_election.cc:304] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f825e4d993594c96a14fcb1a1c742ef2; no voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d
I20260709 13:04:34.484220 19535 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.490727 19535 raft_consensus.cc:2749] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.500007 19360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:48472
I20260709 13:04:34.500754 19360 raft_consensus.cc:606] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS ed29f8586d164987b3531a2710239e3d
I20260709 13:04:34.511842 19360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:48472
I20260709 13:04:34.512389 19360 raft_consensus.cc:606] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS ed29f8586d164987b3531a2710239e3d
I20260709 13:04:34.538635 19529 consensus_queue.cc:1048] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.549103 19479 consensus_queue.cc:1048] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.595288 19539 raft_consensus.cc:993] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:34.595788 19491 raft_consensus.cc:1081] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:34.597453 19286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:36286
I20260709 13:04:34.597957 19286 raft_consensus.cc:493] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.598246 19286 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.603228 19286 raft_consensus.cc:515] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:34.604861 19286 leader_election.cc:290] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.605790 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:34.605904 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96e9a71181b4523b82e78bc0e093f5e" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.606423 19360 raft_consensus.cc:3055] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.606547 19434 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.606758 19360 raft_consensus.cc:740] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.607584 19360 consensus_queue.cc:260] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:34.608536 19360 raft_consensus.cc:3060] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.612250 19434 raft_consensus.cc:2468] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:34.613705 19250 leader_election.cc:304] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:34.614602 19497 raft_consensus.cc:2804] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.614830 19360 raft_consensus.cc:2468] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:34.615139 19497 raft_consensus.cc:697] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:34.615547 19491 consensus_queue.cc:1048] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.616098 19497 consensus_queue.cc:237] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } }
I20260709 13:04:34.625888 19181 catalog_manager.cc:5697] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 2 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.627343 19491 consensus_queue.cc:1048] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.754468 19491 consensus_queue.cc:1048] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.756119 19491 consensus_queue.cc:1048] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.763715 19483 consensus_queue.cc:1048] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.773973 19491 consensus_queue.cc:1048] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.786659 19483 raft_consensus.cc:993] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:34.787163 19483 raft_consensus.cc:1081] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:34.790304 19285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:36286
I20260709 13:04:34.790910 19285 raft_consensus.cc:493] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:34.791327 19285 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.798405 19285 raft_consensus.cc:515] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.800184 19285 leader_election.cc:290] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:34.801069 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:34.801275 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321f8b0faf5b4e4a88fab2da1e36f384" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:34.801800 19360 raft_consensus.cc:3055] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:34.801827 19435 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.802210 19360 raft_consensus.cc:740] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:34.803105 19360 consensus_queue.cc:260] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.804391 19360 raft_consensus.cc:3060] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:34.809859 19435 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:34.811533 19250 leader_election.cc:304] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:34.811606 19360 raft_consensus.cc:2468] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:34.812392 19497 raft_consensus.cc:2804] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:34.812935 19497 raft_consensus.cc:697] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:34.813966 19497 consensus_queue.cc:237] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:34.820708 19181 catalog_manager.cc:5697] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 2 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.097718 19435 raft_consensus.cc:1275] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.099273 19497 consensus_queue.cc:1048] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:35.104090 19360 raft_consensus.cc:1275] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.105525 19497 consensus_queue.cc:1048] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:35.327687 19360 raft_consensus.cc:1275] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.329196 19497 consensus_queue.cc:1048] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:35.336524 19435 raft_consensus.cc:1275] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:35.338231 19552 consensus_queue.cc:1048] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:37.558795 19183 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42116:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:37.560950 19183 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:37.610839 19269 tablet_service.cc:1511] Processing CreateTablet for tablet c8001d125aee47f3bf74213553775dfe (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:37.612301 19269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8001d125aee47f3bf74213553775dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.612404 19348 tablet_service.cc:1511] Processing CreateTablet for tablet c8001d125aee47f3bf74213553775dfe (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:37.613945 19348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8001d125aee47f3bf74213553775dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.615799 19424 tablet_service.cc:1511] Processing CreateTablet for tablet c8001d125aee47f3bf74213553775dfe (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:37.616705 19275 tablet_service.cc:1511] Processing CreateTablet for tablet 478d1bf765e44df48a5fe76d66666df2 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:37.617919 19424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8001d125aee47f3bf74213553775dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.618738 19275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478d1bf765e44df48a5fe76d66666df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.625515 19350 tablet_service.cc:1511] Processing CreateTablet for tablet 478d1bf765e44df48a5fe76d66666df2 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:37.626972 19350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478d1bf765e44df48a5fe76d66666df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.626897 19418 tablet_service.cc:1511] Processing CreateTablet for tablet 478d1bf765e44df48a5fe76d66666df2 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:37.628391 19418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 478d1bf765e44df48a5fe76d66666df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.631623 19346 tablet_service.cc:1511] Processing CreateTablet for tablet 1716ef34dd36473896561ae661766ff4 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:37.632018 19422 tablet_service.cc:1511] Processing CreateTablet for tablet 1716ef34dd36473896561ae661766ff4 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:37.633091 19346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1716ef34dd36473896561ae661766ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.633574 19422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1716ef34dd36473896561ae661766ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.632018 19272 tablet_service.cc:1511] Processing CreateTablet for tablet 1716ef34dd36473896561ae661766ff4 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:37.635555 19268 tablet_service.cc:1511] Processing CreateTablet for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:37.636813 19272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1716ef34dd36473896561ae661766ff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.637871 19347 tablet_service.cc:1511] Processing CreateTablet for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:37.638401 19420 tablet_service.cc:1511] Processing CreateTablet for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:37.639254 19347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.639909 19420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.641865 19268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce6554a7e2dd4bf7a9c66e23c5dc2391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.647183 19270 tablet_service.cc:1511] Processing CreateTablet for tablet f05e331d886243688815b7936d4a9394 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:37.648586 19270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05e331d886243688815b7936d4a9394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.657544 19345 tablet_service.cc:1511] Processing CreateTablet for tablet f05e331d886243688815b7936d4a9394 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:37.659404 19345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05e331d886243688815b7936d4a9394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.672786 19425 tablet_service.cc:1511] Processing CreateTablet for tablet f05e331d886243688815b7936d4a9394 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:37.676813 19271 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4112f7e95442a8908cdb0100f4bf29 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:37.678237 19271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4112f7e95442a8908cdb0100f4bf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.684381 19425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05e331d886243688815b7936d4a9394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.697080 19349 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4112f7e95442a8908cdb0100f4bf29 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:37.698608 19349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4112f7e95442a8908cdb0100f4bf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.703284 19421 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4112f7e95442a8908cdb0100f4bf29 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:37.704679 19421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4112f7e95442a8908cdb0100f4bf29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.706319 19566 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:37.741017 19566 tablet_bootstrap.cc:654] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.742342 19268 tablet_service.cc:1511] Processing CreateTablet for tablet 026310e07db848728601d2b78891d911 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:37.743654 19350 tablet_service.cc:1511] Processing CreateTablet for tablet 026310e07db848728601d2b78891d911 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:37.746187 19350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026310e07db848728601d2b78891d911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.749079 19268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026310e07db848728601d2b78891d911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.750227 19565 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:37.754689 19421 tablet_service.cc:1511] Processing CreateTablet for tablet 026310e07db848728601d2b78891d911 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:37.756179 19421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026310e07db848728601d2b78891d911. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:37.758298 19232 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:37.764540 19567 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:37.765785 19351 tablet_service.cc:1511] Processing CreateTablet for tablet 570e44dbcfd141aa9aeec09ad7e06f51 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:37.767335 19351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570e44dbcfd141aa9aeec09ad7e06f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.772382 19420 tablet_service.cc:1511] Processing CreateTablet for tablet 570e44dbcfd141aa9aeec09ad7e06f51 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:37.773841 19420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570e44dbcfd141aa9aeec09ad7e06f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.777875 19567 tablet_bootstrap.cc:654] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.783819 19565 tablet_bootstrap.cc:654] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.764540 19275 tablet_service.cc:1511] Processing CreateTablet for tablet 570e44dbcfd141aa9aeec09ad7e06f51 (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:37.790030 19275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 570e44dbcfd141aa9aeec09ad7e06f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.793517 19566 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:37.803795 19566 ts_tablet_manager.cc:1403] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.003s
I20260709 13:04:37.794262 19271 tablet_service.cc:1511] Processing CreateTablet for tablet 3c26262294aa42a1ae16a744c7e6907e (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:37.798354 19345 tablet_service.cc:1511] Processing CreateTablet for tablet 3c26262294aa42a1ae16a744c7e6907e (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:37.806298 19345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c26262294aa42a1ae16a744c7e6907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.806521 19271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c26262294aa42a1ae16a744c7e6907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.799844 19425 tablet_service.cc:1511] Processing CreateTablet for tablet 3c26262294aa42a1ae16a744c7e6907e (DEFAULT_TABLE table=soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:37.808140 19425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c26262294aa42a1ae16a744c7e6907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.818935 19566 raft_consensus.cc:359] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.820045 19565 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:37.821951 19566 raft_consensus.cc:385] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.822247 19565 ts_tablet_manager.cc:1403] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.004s
I20260709 13:04:37.829844 19565 raft_consensus.cc:359] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.830667 19565 raft_consensus.cc:385] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.830956 19565 raft_consensus.cc:740] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.831929 19567 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:37.832532 19567 ts_tablet_manager.cc:1403] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.068s	user 0.006s	sys 0.019s
I20260709 13:04:37.832594 19565 consensus_queue.cc:260] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.840385 19565 ts_tablet_manager.cc:1434] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:37.841586 19565 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:37.841917 19566 raft_consensus.cc:740] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.844960 19567 raft_consensus.cc:359] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.846989 19567 raft_consensus.cc:385] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.847565 19567 raft_consensus.cc:740] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.848464 19567 consensus_queue.cc:260] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.852902 19567 ts_tablet_manager.cc:1434] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:04:37.854060 19567 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:37.843502 19566 consensus_queue.cc:260] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.858162 19565 tablet_bootstrap.cc:654] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.862308 19552 raft_consensus.cc:493] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.860037 19567 tablet_bootstrap.cc:654] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.862893 19552 raft_consensus.cc:515] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.865916 19552 leader_election.cc:290] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:37.867722 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:37.868824 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:37.870566 19484 raft_consensus.cc:493] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.874819 19250 leader_election.cc:343] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.871384 19566 ts_tablet_manager.cc:1434] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.001s
W20260709 13:04:37.877416 19251 leader_election.cc:343] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.874935 19484 raft_consensus.cc:515] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.877969 19251 leader_election.cc:304] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed29f8586d164987b3531a2710239e3d; no voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:37.875379 19567 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:37.879231 19552 raft_consensus.cc:2749] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:37.879431 19567 ts_tablet_manager.cc:1403] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.003s
I20260709 13:04:37.880760 19566 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:37.882417 19484 leader_election.cc:290] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:37.882467 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
W20260709 13:04:37.886118 19400 leader_election.cc:343] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.884243 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
W20260709 13:04:37.888154 19400 leader_election.cc:343] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.888607 19400 leader_election.cc:304] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f825e4d993594c96a14fcb1a1c742ef2; no voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d
I20260709 13:04:37.889210 19566 tablet_bootstrap.cc:654] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.889777 19484 raft_consensus.cc:2749] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:37.891497 19567 raft_consensus.cc:359] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.892273 19567 raft_consensus.cc:385] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.892586 19567 raft_consensus.cc:740] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.893353 19567 consensus_queue.cc:260] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.894898 19565 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:37.895574 19565 ts_tablet_manager.cc:1403] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.011s
I20260709 13:04:37.896109 19567 ts_tablet_manager.cc:1434] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:04:37.897322 19567 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:37.898907 19565 raft_consensus.cc:359] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.899750 19565 raft_consensus.cc:385] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.900112 19565 raft_consensus.cc:740] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.900894 19565 consensus_queue.cc:260] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.904553 19567 tablet_bootstrap.cc:654] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.924404 19565 ts_tablet_manager.cc:1434] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
W20260709 13:04:37.924110 19251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.130:38063, user_credentials={real_user=slave}} blocked reactor thread for 46756.2us
I20260709 13:04:37.925835 19565 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:37.937071 19566 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:37.938110 19566 ts_tablet_manager.cc:1403] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.023s
I20260709 13:04:37.938918 19565 tablet_bootstrap.cc:654] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.940686 19567 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:37.941329 19567 ts_tablet_manager.cc:1403] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.012s
I20260709 13:04:37.941668 19566 raft_consensus.cc:359] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.942502 19566 raft_consensus.cc:385] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.942816 19566 raft_consensus.cc:740] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.943723 19566 consensus_queue.cc:260] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.944638 19567 raft_consensus.cc:359] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.945442 19567 raft_consensus.cc:385] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.945719 19567 raft_consensus.cc:740] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.946491 19567 consensus_queue.cc:260] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.948794 19567 ts_tablet_manager.cc:1434] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:37.951249 19566 ts_tablet_manager.cc:1434] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 13:04:37.952359 19566 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:37.954126 19565 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:37.954722 19565 ts_tablet_manager.cc:1403] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.010s
I20260709 13:04:37.955380 19567 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:37.958123 19565 raft_consensus.cc:359] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.958958 19565 raft_consensus.cc:385] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.959261 19566 tablet_bootstrap.cc:654] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.959291 19565 raft_consensus.cc:740] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.960562 19565 consensus_queue.cc:260] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.962473 19567 tablet_bootstrap.cc:654] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.964181 19565 ts_tablet_manager.cc:1434] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:04:37.970366 19565 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:37.979173 19565 tablet_bootstrap.cc:654] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.983047 19566 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:37.983711 19566 ts_tablet_manager.cc:1403] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.011s
I20260709 13:04:37.983947 19567 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:37.984495 19567 ts_tablet_manager.cc:1403] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.015s
I20260709 13:04:37.986776 19574 raft_consensus.cc:493] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.987241 19566 raft_consensus.cc:359] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.987496 19567 raft_consensus.cc:359] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.988070 19566 raft_consensus.cc:385] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.987391 19574 raft_consensus.cc:515] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.988410 19567 raft_consensus.cc:385] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.988461 19566 raft_consensus.cc:740] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.988839 19567 raft_consensus.cc:740] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.989418 19566 consensus_queue.cc:260] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.989670 19567 consensus_queue.cc:260] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:37.992183 19566 ts_tablet_manager.cc:1434] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:37.992686 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:37.993462 19566 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:37.994073 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:37.994354 19567 ts_tablet_manager.cc:1434] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:04:37.994885 19360 raft_consensus.cc:2468] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:37.995481 19567 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:37.996417 19251 leader_election.cc:304] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:37.997373 19552 raft_consensus.cc:2804] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:37.997824 19552 raft_consensus.cc:493] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:37.998008 19250 leader_election.cc:343] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.998191 19552 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.998816 19574 leader_election.cc:290] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.002825 19567 tablet_bootstrap.cc:654] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.006050 19565 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.006603 19565 ts_tablet_manager.cc:1403] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.006s
I20260709 13:04:38.007061 19552 raft_consensus.cc:515] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.010182 19565 raft_consensus.cc:359] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.011075 19565 raft_consensus.cc:385] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.011612 19565 raft_consensus.cc:740] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.012603 19565 consensus_queue.cc:260] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.015509 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.015949 19552 leader_election.cc:290] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.016294 19360 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.016970 19566 tablet_bootstrap.cc:654] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.022730 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
W20260709 13:04:38.024477 19250 leader_election.cc:343] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.025105 19567 tablet_bootstrap.cc:492] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:38.025688 19567 ts_tablet_manager.cc:1403] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.006s
I20260709 13:04:38.026371 19566 tablet_bootstrap.cc:492] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.026935 19566 ts_tablet_manager.cc:1403] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.000s
I20260709 13:04:38.028661 19565 ts_tablet_manager.cc:1434] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:04:38.028839 19567 raft_consensus.cc:359] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.029094 19360 raft_consensus.cc:2468] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:38.029737 19567 raft_consensus.cc:385] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.030575 19565 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:38.030314 19566 raft_consensus.cc:359] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.030802 19567 raft_consensus.cc:740] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.031350 19566 raft_consensus.cc:385] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.031704 19566 raft_consensus.cc:740] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.031749 19251 leader_election.cc:304] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:38.032102 19567 consensus_queue.cc:260] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.032549 19566 consensus_queue.cc:260] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.033231 19552 raft_consensus.cc:2804] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.034065 19552 raft_consensus.cc:697] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:38.035327 19566 ts_tablet_manager.cc:1434] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:38.035070 19552 consensus_queue.cc:237] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.036007 19567 ts_tablet_manager.cc:1434] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:04:38.036439 19566 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:38.037112 19567 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:38.034142 19573 raft_consensus.cc:493] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.037885 19573 raft_consensus.cc:515] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.039971 19565 tablet_bootstrap.cc:654] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.041899 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:38.042663 19435 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:38.043864 19566 tablet_bootstrap.cc:654] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.044315 19567 tablet_bootstrap.cc:654] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.044212 19325 leader_election.cc:304] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.045712 19483 raft_consensus.cc:2804] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.046190 19483 raft_consensus.cc:493] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.046648 19483 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.049808 19573 leader_election.cc:290] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.057642 19566 tablet_bootstrap.cc:492] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.058259 19566 ts_tablet_manager.cc:1403] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.008s
I20260709 13:04:38.061213 19567 tablet_bootstrap.cc:492] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:38.062036 19567 ts_tablet_manager.cc:1403] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.010s
I20260709 13:04:38.062664 19566 raft_consensus.cc:359] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.063539 19566 raft_consensus.cc:385] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.100956 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.103971 19285 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:38.112767 19484 raft_consensus.cc:493] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.114336 19484 raft_consensus.cc:515] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.063889 19566 raft_consensus.cc:740] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.115370 19574 raft_consensus.cc:493] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.121166 19566 consensus_queue.cc:260] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.122793 19574 raft_consensus.cc:515] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.123042 19483 raft_consensus.cc:515] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.124069 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.124861 19285 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.126282 19400 leader_election.cc:304] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.126808 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.126876 19483 leader_election.cc:290] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.127806 19286 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.129822 19484 leader_election.cc:290] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.130957 19484 raft_consensus.cc:2804] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.131436 19484 raft_consensus.cc:493] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.131840 19484 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.135322 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:38.136577 19435 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.137137 19483 raft_consensus.cc:493] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.138432 19574 leader_election.cc:290] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.138051 19483 raft_consensus.cc:515] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.142078 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.142678 19483 leader_election.cc:290] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.143299 19360 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:38.143741 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
W20260709 13:04:38.145785 19326 leader_election.cc:343] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.146324 19434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:38.141000 19565 tablet_bootstrap.cc:492] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.150156 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:38.151530 19565 ts_tablet_manager.cc:1403] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.121s	user 0.013s	sys 0.020s
I20260709 13:04:38.151793 19433 raft_consensus.cc:2468] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:38.154192 19325 leader_election.cc:304] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: ed29f8586d164987b3531a2710239e3d
I20260709 13:04:38.155444 19483 raft_consensus.cc:2804] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.155851 19483 raft_consensus.cc:493] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.150259 19251 leader_election.cc:304] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:38.156636 19483 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.161520 19574 raft_consensus.cc:2804] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.161983 19574 raft_consensus.cc:493] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.162405 19574 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.172175 19483 raft_consensus.cc:515] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.179481 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.180279 19566 ts_tablet_manager.cc:1434] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.122s	user 0.010s	sys 0.023s
W20260709 13:04:38.186476 19400 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:44435, user_credentials={real_user=slave}} blocked reactor thread for 60434.5us
I20260709 13:04:38.182718 19567 raft_consensus.cc:359] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.190472 19567 raft_consensus.cc:385] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.190917 19567 raft_consensus.cc:740] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.191275 19566 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:38.191294 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.192327 19360 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.192026 19567 consensus_queue.cc:260] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.176874 19565 raft_consensus.cc:359] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.194175 19565 raft_consensus.cc:385] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.188755 19286 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.194592 19565 raft_consensus.cc:740] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.196058 19326 leader_election.cc:304] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:38.197160 19483 leader_election.cc:290] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.197217 19573 raft_consensus.cc:2804] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.197999 19573 raft_consensus.cc:697] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:38.197885 19565 consensus_queue.cc:260] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.198980 19285 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.199609 19567 ts_tablet_manager.cc:1434] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.136s	user 0.003s	sys 0.009s
I20260709 13:04:38.199342 19573 consensus_queue.cc:237] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.200793 19565 ts_tablet_manager.cc:1434] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.007s
I20260709 13:04:38.198396 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:38.198509 19484 raft_consensus.cc:515] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.201638 19433 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.202554 19565 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:38.202565 19434 raft_consensus.cc:2393] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed29f8586d164987b3531a2710239e3d in current term 1: Already voted for candidate f825e4d993594c96a14fcb1a1c742ef2 in this term.
I20260709 13:04:38.201514 19567 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:38.208650 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.211108 19484 leader_election.cc:290] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.212313 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.212954 19360 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.215497 19566 tablet_bootstrap.cc:654] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.217840 19567 tablet_bootstrap.cc:654] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.225966 19433 raft_consensus.cc:2468] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.227980 19325 leader_election.cc:304] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.229171 19483 raft_consensus.cc:2804] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.230299 19285 raft_consensus.cc:2468] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.238507 19435 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.239576 19184 catalog_manager.cc:5697] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.245393 19573 raft_consensus.cc:493] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.246111 19573 raft_consensus.cc:515] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.249441 19573 leader_election.cc:290] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.249838 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
W20260709 13:04:38.251763 19325 leader_election.cc:343] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.256856 19483 raft_consensus.cc:697] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:38.257036 19565 tablet_bootstrap.cc:654] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.255780 19574 raft_consensus.cc:515] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.260723 19566 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.261340 19566 ts_tablet_manager.cc:1403] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.001s
I20260709 13:04:38.258016 19483 consensus_queue.cc:237] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.264586 19360 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.266434 19286 raft_consensus.cc:2393] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f825e4d993594c96a14fcb1a1c742ef2 in current term 1: Already voted for candidate ed29f8586d164987b3531a2710239e3d in this term.
I20260709 13:04:38.265240 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.265509 19566 raft_consensus.cc:359] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.269475 19567 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
W20260709 13:04:38.269819 19326 leader_election.cc:343] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.269982 19566 raft_consensus.cc:385] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.270222 19567 ts_tablet_manager.cc:1403] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.012s
I20260709 13:04:38.270368 19326 leader_election.cc:304] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786; no voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:38.270656 19566 raft_consensus.cc:740] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.271783 19566 consensus_queue.cc:260] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.272007 19593 raft_consensus.cc:2749] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.274674 19400 leader_election.cc:304] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: ed29f8586d164987b3531a2710239e3d
I20260709 13:04:38.276098 19589 raft_consensus.cc:2804] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.276561 19589 raft_consensus.cc:697] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:38.276844 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:38.280169 19574 leader_election.cc:290] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.276823 19567 raft_consensus.cc:359] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.281629 19567 raft_consensus.cc:385] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.281939 19567 raft_consensus.cc:740] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.281853 19589 consensus_queue.cc:237] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.283090 19567 consensus_queue.cc:260] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.287384 19567 ts_tablet_manager.cc:1434] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:04:38.289119 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.290264 19360 raft_consensus.cc:2393] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed29f8586d164987b3531a2710239e3d in current term 1: Already voted for candidate f825e4d993594c96a14fcb1a1c742ef2 in this term.
I20260709 13:04:38.290709 19566 ts_tablet_manager.cc:1434] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:04:38.291929 19251 leader_election.cc:304] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed29f8586d164987b3531a2710239e3d; no voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:38.292630 19566 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:38.293769 19574 raft_consensus.cc:2749] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.297897 19484 raft_consensus.cc:493] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.298626 19484 raft_consensus.cc:515] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.302306 19484 leader_election.cc:290] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.303529 19567 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:38.304080 19566 tablet_bootstrap.cc:654] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.303843 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.303885 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.307022 19360 raft_consensus.cc:2393] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f825e4d993594c96a14fcb1a1c742ef2 in current term 1: Already voted for candidate ed29f8586d164987b3531a2710239e3d in this term.
I20260709 13:04:38.308632 19400 leader_election.cc:304] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f825e4d993594c96a14fcb1a1c742ef2; no voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d
I20260709 13:04:38.310139 19589 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.313943 19565 tablet_bootstrap.cc:492] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.314874 19565 ts_tablet_manager.cc:1403] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.010s
I20260709 13:04:38.313112 19567 tablet_bootstrap.cc:654] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.326001 19565 raft_consensus.cc:359] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.327167 19565 raft_consensus.cc:385] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.327591 19565 raft_consensus.cc:740] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.328678 19565 consensus_queue.cc:260] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.331794 19565 ts_tablet_manager.cc:1434] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:04:38.335062 19566 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.335645 19566 ts_tablet_manager.cc:1403] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.016s
I20260709 13:04:38.336344 19484 raft_consensus.cc:493] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.336023 19181 catalog_manager.cc:5697] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 1 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.336846 19484 raft_consensus.cc:515] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.338933 19484 leader_election.cc:290] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.338964 19566 raft_consensus.cc:359] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.339831 19566 raft_consensus.cc:385] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.340117 19566 raft_consensus.cc:740] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.340876 19566 consensus_queue.cc:260] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.328696 19589 raft_consensus.cc:2749] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.333171 19565 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:38.343039 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.343322 19566 ts_tablet_manager.cc:1434] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:38.343756 19360 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.340883 19285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.344718 19566 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:38.345223 19285 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.346789 19400 leader_election.cc:304] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.347692 19589 raft_consensus.cc:2804] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.348101 19589 raft_consensus.cc:493] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.348448 19589 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.350255 19565 tablet_bootstrap.cc:654] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.352762 19566 tablet_bootstrap.cc:654] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.357929 19567 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:38.358563 19567 ts_tablet_manager.cc:1403] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.001s
I20260709 13:04:38.359589 19184 catalog_manager.cc:5697] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.362291 19567 raft_consensus.cc:359] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.364625 19566 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.367535 19566 ts_tablet_manager.cc:1403] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.012s
I20260709 13:04:38.370615 19566 raft_consensus.cc:359] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.371289 19566 raft_consensus.cc:385] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.371634 19566 raft_consensus.cc:740] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.372391 19566 consensus_queue.cc:260] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.374405 19566 ts_tablet_manager.cc:1434] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:38.375319 19566 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: Bootstrap starting.
I20260709 13:04:38.371398 19567 raft_consensus.cc:385] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.377645 19567 raft_consensus.cc:740] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.378614 19567 consensus_queue.cc:260] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.380846 19566 tablet_bootstrap.cc:654] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.383076 19567 ts_tablet_manager.cc:1434] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:04:38.384447 19567 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Bootstrap starting.
I20260709 13:04:38.383539 19182 catalog_manager.cc:5697] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.366051 19589 raft_consensus.cc:515] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.389734 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.390617 19286 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.395571 19566 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: No bootstrap required, opened a new log
I20260709 13:04:38.396131 19566 ts_tablet_manager.cc:1403] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.010s
I20260709 13:04:38.398383 19589 leader_election.cc:290] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.398649 19286 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.399084 19566 raft_consensus.cc:359] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.399544 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.399881 19400 leader_election.cc:304] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.400230 19360 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.400799 19589 raft_consensus.cc:2804] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.401337 19589 raft_consensus.cc:697] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:38.402256 19589 consensus_queue.cc:237] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.412668 19565 tablet_bootstrap.cc:492] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.413710 19360 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.414143 19567 tablet_bootstrap.cc:654] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.399868 19566 raft_consensus.cc:385] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.416746 19566 raft_consensus.cc:740] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.417582 19566 consensus_queue.cc:260] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.414167 19565 ts_tablet_manager.cc:1403] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.081s	user 0.035s	sys 0.023s
I20260709 13:04:38.420913 19566 ts_tablet_manager.cc:1434] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:04:38.421736 19565 raft_consensus.cc:359] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.422600 19565 raft_consensus.cc:385] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.422932 19565 raft_consensus.cc:740] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.423756 19565 consensus_queue.cc:260] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.426308 19565 ts_tablet_manager.cc:1434] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:38.427692 19565 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:38.428128 19567 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: No bootstrap required, opened a new log
I20260709 13:04:38.428654 19567 ts_tablet_manager.cc:1403] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.006s
I20260709 13:04:38.431653 19567 raft_consensus.cc:359] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.432374 19567 raft_consensus.cc:385] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.432650 19567 raft_consensus.cc:740] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.433374 19567 consensus_queue.cc:260] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.434984 19565 tablet_bootstrap.cc:654] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.435591 19567 ts_tablet_manager.cc:1434] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:04:38.443663 19565 tablet_bootstrap.cc:492] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.444156 19565 ts_tablet_manager.cc:1403] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:04:38.446833 19574 raft_consensus.cc:493] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.447165 19565 raft_consensus.cc:359] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.447288 19574 raft_consensus.cc:515] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.447952 19565 raft_consensus.cc:385] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.448304 19565 raft_consensus.cc:740] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.449038 19565 consensus_queue.cc:260] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.450659 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:38.451176 19565 ts_tablet_manager.cc:1434] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:38.451363 19435 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:38.452433 19565 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: Bootstrap starting.
I20260709 13:04:38.453545 19182 catalog_manager.cc:5697] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.455473 19250 leader_election.cc:304] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.454313 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.456943 19361 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 0.
I20260709 13:04:38.460983 19565 tablet_bootstrap.cc:654] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.462172 19609 raft_consensus.cc:2804] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.468115 19609 raft_consensus.cc:493] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.468619 19609 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.472822 19589 raft_consensus.cc:493] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.473645 19589 raft_consensus.cc:515] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.474646 19574 leader_election.cc:290] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.476615 19589 leader_election.cc:290] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.478490 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.479207 19286 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.481474 19400 leader_election.cc:304] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.483327 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.484134 19360 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.484140 19589 raft_consensus.cc:2804] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.484835 19589 raft_consensus.cc:493] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.485193 19589 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.488718 19609 raft_consensus.cc:515] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.492061 19589 raft_consensus.cc:515] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.492122 19609 leader_election.cc:290] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.494634 19565 tablet_bootstrap.cc:492] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: No bootstrap required, opened a new log
I20260709 13:04:38.495245 19565 ts_tablet_manager.cc:1403] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.016s
I20260709 13:04:38.495175 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.496121 19286 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.497210 19589 leader_election.cc:290] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.497220 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.498088 19360 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.498558 19565 raft_consensus.cc:359] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.499269 19565 raft_consensus.cc:385] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.499337 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.499596 19565 raft_consensus.cc:740] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.500233 19361 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.500522 19565 consensus_queue.cc:260] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.502828 19565 ts_tablet_manager.cc:1434] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:38.504083 19286 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.505359 19400 leader_election.cc:304] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.506089 19360 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:38.506415 19589 raft_consensus.cc:2804] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.507009 19589 raft_consensus.cc:697] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:38.512437 19361 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.514840 19251 leader_election.cc:304] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:38.515772 19552 raft_consensus.cc:2804] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.516331 19552 raft_consensus.cc:697] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:38.518189 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:38.518841 19435 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.517198 19552 consensus_queue.cc:237] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.507880 19589 consensus_queue.cc:237] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.521128 19574 consensus_queue.cc:1048] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:04:38.527442 19574 consensus_queue.cc:1048] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.528576 19435 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 1.
I20260709 13:04:38.545006 19183 catalog_manager.cc:5697] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.541478 19181 catalog_manager.cc:5697] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 1 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.588637 19593 consensus_queue.cc:1048] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.599669 19483 consensus_queue.cc:1048] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.682726 19483 raft_consensus.cc:493] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.683156 19483 raft_consensus.cc:515] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.684736 19483 leader_election.cc:290] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.685590 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.685853 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2" is_pre_election: true
I20260709 13:04:38.686216 19286 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:38.686635 19433 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 0.
I20260709 13:04:38.687234 19326 leader_election.cc:304] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:38.688052 19573 raft_consensus.cc:2804] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.688341 19573 raft_consensus.cc:493] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.688578 19573 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.694854 19573 raft_consensus.cc:515] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.696481 19483 consensus_queue.cc:1048] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.696995 19573 leader_election.cc:290] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:38.697813 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.698354 19286 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.698136 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:38.699008 19433 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.706110 19286 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.707562 19433 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 1.
I20260709 13:04:38.707623 19326 leader_election.cc:304] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:38.708830 19573 raft_consensus.cc:2804] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.709309 19573 raft_consensus.cc:697] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:38.710167 19573 consensus_queue.cc:237] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.712812 19483 consensus_queue.cc:1048] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.725731 19484 consensus_queue.cc:1048] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:38.733983 19181 catalog_manager.cc:5697] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 1 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.749768 19628 consensus_queue.cc:1048] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:38.766894 19232 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:38.811062 19589 raft_consensus.cc:493] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.811721 19589 raft_consensus.cc:515] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.813604 19589 leader_election.cc:290] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.814679 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d" is_pre_election: true
I20260709 13:04:38.815029 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786" is_pre_election: true
I20260709 13:04:38.815362 19286 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.815793 19360 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 0.
I20260709 13:04:38.816612 19400 leader_election.cc:304] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.817461 19589 raft_consensus.cc:2804] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.817819 19589 raft_consensus.cc:493] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.818133 19589 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.821420 19628 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.828058 19589 raft_consensus.cc:515] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.836130 19628 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:38.837414 19589 leader_election.cc:290] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:38.839293 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:38.840018 19286 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.842625 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:38.843338 19360 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.848553 19286 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.849860 19400 leader_election.cc:304] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:38.850692 19484 raft_consensus.cc:2804] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.851245 19484 raft_consensus.cc:697] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:38.852166 19484 consensus_queue.cc:237] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:38.852181 19360 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 1.
I20260709 13:04:38.863447 19181 catalog_manager.cc:5697] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 1 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.907423 19484 consensus_queue.cc:1048] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.914290 19574 consensus_queue.cc:1048] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.918164 19484 consensus_queue.cc:1048] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.940886 19574 consensus_queue.cc:1048] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.090960 19573 consensus_queue.cc:1048] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.101027 19623 consensus_queue.cc:1048] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.141693 19361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:48472
I20260709 13:04:39.142267 19361 raft_consensus.cc:606] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS ed29f8586d164987b3531a2710239e3d
I20260709 13:04:39.147161 19361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:48472
I20260709 13:04:39.147634 19361 raft_consensus.cc:606] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS ed29f8586d164987b3531a2710239e3d
I20260709 13:04:39.163251 19435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1716ef34dd36473896561ae661766ff4"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
mode: GRACEFUL
new_leader_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:35856
I20260709 13:04:39.163739 19435 raft_consensus.cc:606] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Received request to transfer leadership to TS ed29f8586d164987b3531a2710239e3d
I20260709 13:04:39.171643 19435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f05e331d886243688815b7936d4a9394"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
mode: GRACEFUL
new_leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:35856
I20260709 13:04:39.172061 19435 raft_consensus.cc:606] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Received request to transfer leadership to TS c4a3067ebfa947e18a871d0ac65c2786
I20260709 13:04:39.182375 19573 raft_consensus.cc:993] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.182876 19483 raft_consensus.cc:1081] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.184381 19286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:36286
I20260709 13:04:39.184998 19286 raft_consensus.cc:493] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.185406 19286 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.188508 19285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
mode: GRACEFUL
new_leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:36304
I20260709 13:04:39.189208 19285 raft_consensus.cc:606] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Received request to transfer leadership to TS c4a3067ebfa947e18a871d0ac65c2786
I20260709 13:04:39.193189 19286 raft_consensus.cc:515] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.194511 19631 raft_consensus.cc:993] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: : Instructing follower ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.195055 19631 raft_consensus.cc:1081] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Signalling peer ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.195785 19435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "026310e07db848728601d2b78891d911"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
mode: GRACEFUL
new_leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:35856
I20260709 13:04:39.196378 19435 raft_consensus.cc:606] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Received request to transfer leadership to TS c4a3067ebfa947e18a871d0ac65c2786
I20260709 13:04:39.196762 19284 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1716ef34dd36473896561ae661766ff4"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:36296
I20260709 13:04:39.197381 19284 raft_consensus.cc:493] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.197491 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:39.197794 19284 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.198208 19361 raft_consensus.cc:3055] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.198576 19361 raft_consensus.cc:740] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.199359 19361 consensus_queue.cc:260] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.200233 19361 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.206153 19286 leader_election.cc:290] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.206135 19284 raft_consensus.cc:515] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.206331 19435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8001d125aee47f3bf74213553775dfe" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.207388 19435 raft_consensus.cc:3060] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.207526 19433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
mode: GRACEFUL
new_leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:35856
I20260709 13:04:39.208221 19433 raft_consensus.cc:606] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Received request to transfer leadership to TS c4a3067ebfa947e18a871d0ac65c2786
I20260709 13:04:39.209029 19361 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.210330 19251 leader_election.cc:304] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:39.211196 19611 raft_consensus.cc:2804] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.211767 19611 raft_consensus.cc:697] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:39.212581 19611 consensus_queue.cc:237] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.214620 19361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:48472
I20260709 13:04:39.215302 19361 raft_consensus.cc:606] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:39.216485 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:39.216805 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1716ef34dd36473896561ae661766ff4" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.217065 19361 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.217547 19433 raft_consensus.cc:3055] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.217882 19433 raft_consensus.cc:740] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:39.218699 19433 consensus_queue.cc:260] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.219871 19433 raft_consensus.cc:3060] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.221447 19284 leader_election.cc:290] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.221696 19181 catalog_manager.cc:5697] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 2 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.223986 19435 raft_consensus.cc:2468] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.224748 19361 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.229615 19251 leader_election.cc:304] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:39.230479 19611 raft_consensus.cc:2804] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.231038 19433 raft_consensus.cc:2468] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.233328 19611 raft_consensus.cc:697] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:39.234427 19611 consensus_queue.cc:237] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.240891 19483 raft_consensus.cc:993] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.241425 19573 raft_consensus.cc:1081] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer ed29f8586d164987b3531a2710239e3d to start an election
I20260709 13:04:39.243057 19284 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
 from {username='slave'} at 127.0.0.1:36286
I20260709 13:04:39.243642 19284 raft_consensus.cc:493] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.244027 19284 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.245051 19183 catalog_manager.cc:5697] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 1 to 2, leader changed from f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131) to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 2 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.250361 19284 raft_consensus.cc:515] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.252511 19284 leader_election.cc:290] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Requested vote from peers c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.253613 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:39.253998 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "478d1bf765e44df48a5fe76d66666df2" candidate_uuid: "ed29f8586d164987b3531a2710239e3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.254230 19360 raft_consensus.cc:3055] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.254555 19360 raft_consensus.cc:740] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.254634 19433 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.255331 19360 consensus_queue.cc:260] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.256541 19360 raft_consensus.cc:3060] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.261862 19433 raft_consensus.cc:2468] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.263187 19250 leader_election.cc:304] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:39.263383 19360 raft_consensus.cc:2468] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed29f8586d164987b3531a2710239e3d in term 2.
I20260709 13:04:39.263986 19611 raft_consensus.cc:2804] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.264508 19611 raft_consensus.cc:697] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Becoming Leader. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:39.265239 19611 consensus_queue.cc:237] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.271591 19183 catalog_manager.cc:5697] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to ed29f8586d164987b3531a2710239e3d (127.7.117.129). New cstate: current_term: 2 leader_uuid: "ed29f8586d164987b3531a2710239e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.344985 19628 raft_consensus.cc:993] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: : Instructing follower c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.345474 19484 raft_consensus.cc:1081] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Signalling peer c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.346853 19360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f05e331d886243688815b7936d4a9394"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:48456
I20260709 13:04:39.347234 19360 raft_consensus.cc:493] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.347448 19360 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.351758 19360 raft_consensus.cc:515] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.353109 19360 leader_election.cc:290] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.354187 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.354099 19284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:39.354911 19433 raft_consensus.cc:3055] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.354984 19284 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.355232 19433 raft_consensus.cc:740] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:39.356072 19433 consensus_queue.cc:260] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.357031 19433 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.360584 19284 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.361899 19326 leader_election.cc:304] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:39.362066 19433 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.362632 19573 raft_consensus.cc:2804] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.363122 19573 raft_consensus.cc:697] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.363903 19573 consensus_queue.cc:237] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.370342 19183 catalog_manager.cc:5697] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 1 to 2, leader changed from f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131) to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.406476 19484 consensus_queue.cc:1048] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.412781 19484 consensus_queue.cc:1048] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.424499 19628 raft_consensus.cc:993] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: : Instructing follower c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.425022 19627 raft_consensus.cc:1081] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Signalling peer c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.426644 19360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:48456
I20260709 13:04:39.427937 19360 raft_consensus.cc:493] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.428260 19360 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.433511 19360 raft_consensus.cc:515] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.435319 19360 leader_election.cc:290] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.436426 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.437124 19433 raft_consensus.cc:3055] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.437495 19433 raft_consensus.cc:740] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:39.438375 19286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "570e44dbcfd141aa9aeec09ad7e06f51" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:39.438298 19433 consensus_queue.cc:260] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.438977 19286 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.439749 19433 raft_consensus.cc:3060] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.446393 19286 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.447592 19433 raft_consensus.cc:2468] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.447597 19326 leader_election.cc:304] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:39.449795 19573 raft_consensus.cc:2804] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.450269 19573 raft_consensus.cc:697] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.451109 19573 consensus_queue.cc:237] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.457911 19181 catalog_manager.cc:5697] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 1 to 2, leader changed from f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131) to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.485785 19628 raft_consensus.cc:993] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: : Instructing follower c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.486258 19628 raft_consensus.cc:1081] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Signalling peer c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.487707 19360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "026310e07db848728601d2b78891d911"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:48456
I20260709 13:04:39.488173 19360 raft_consensus.cc:493] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.488453 19360 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.494590 19360 raft_consensus.cc:515] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.496515 19360 leader_election.cc:290] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.497339 19284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:39.497529 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026310e07db848728601d2b78891d911" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.498031 19284 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.498081 19433 raft_consensus.cc:3055] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.498545 19433 raft_consensus.cc:740] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:39.499378 19433 consensus_queue.cc:260] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.500623 19433 raft_consensus.cc:3060] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.504792 19645 raft_consensus.cc:993] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: : Instructing follower c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.505419 19611 raft_consensus.cc:1081] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Signalling peer c4a3067ebfa947e18a871d0ac65c2786 to start an election
I20260709 13:04:39.505592 19284 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.506956 19326 leader_election.cc:304] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, ed29f8586d164987b3531a2710239e3d; no voters: 
I20260709 13:04:39.507018 19360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
 from {username='slave'} at 127.0.0.1:48452
I20260709 13:04:39.507853 19573 raft_consensus.cc:2804] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.507957 19360 raft_consensus.cc:493] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.507807 19433 raft_consensus.cc:2468] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.508772 19573 raft_consensus.cc:697] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.508991 19360 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.509730 19573 consensus_queue.cc:237] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.515556 19360 raft_consensus.cc:515] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.517669 19360 leader_election.cc:290] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131:38493)
I20260709 13:04:39.519201 19433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
I20260709 13:04:39.519901 19433 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.520958 19284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4112f7e95442a8908cdb0100f4bf29" candidate_uuid: "c4a3067ebfa947e18a871d0ac65c2786" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:39.521584 19284 raft_consensus.cc:3055] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.521963 19284 raft_consensus.cc:740] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:39.521663 19181 catalog_manager.cc:5697] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 1 to 2, leader changed from f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131) to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.523277 19284 consensus_queue.cc:260] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.524560 19284 raft_consensus.cc:3060] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.527158 19433 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.528208 19325 leader_election.cc:304] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:39.528949 19573 raft_consensus.cc:2804] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.529708 19573 raft_consensus.cc:697] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Becoming Leader. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.531322 19284 raft_consensus.cc:2468] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4a3067ebfa947e18a871d0ac65c2786 in term 2.
I20260709 13:04:39.530992 19573 consensus_queue.cc:237] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.538319 19181 catalog_manager.cc:5697] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 reported cstate change: term changed from 1 to 2, leader changed from ed29f8586d164987b3531a2710239e3d (127.7.117.129) to c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130). New cstate: current_term: 2 leader_uuid: "c4a3067ebfa947e18a871d0ac65c2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.597774 19433 raft_consensus.cc:1275] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.599107 19611 consensus_queue.cc:1048] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:39.611515 19361 raft_consensus.cc:1275] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.613086 19574 consensus_queue.cc:1048] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.677414 19433 raft_consensus.cc:1275] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.678709 19574 consensus_queue.cc:1048] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.686761 19361 raft_consensus.cc:1275] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.689826 19574 consensus_queue.cc:1048] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.702481 19483 raft_consensus.cc:993] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:39.702950 19483 raft_consensus.cc:1081] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:39.704638 19433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:35842
I20260709 13:04:39.705174 19433 raft_consensus.cc:493] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.705561 19433 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.710902 19433 raft_consensus.cc:515] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.712828 19433 leader_election.cc:290] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:39.713786 19284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:39.714033 19361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c26262294aa42a1ae16a744c7e6907e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:39.714412 19284 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.714936 19361 raft_consensus.cc:3055] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.715361 19361 raft_consensus.cc:740] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:39.716329 19361 consensus_queue.cc:260] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.717198 19361 raft_consensus.cc:3060] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.720386 19284 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:39.721474 19400 leader_election.cc:304] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:39.722256 19628 raft_consensus.cc:2804] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.722484 19361 raft_consensus.cc:2468] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:39.722754 19628 raft_consensus.cc:697] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:39.723623 19628 consensus_queue.cc:237] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:39.730039 19181 catalog_manager.cc:5697] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.775671 19361 raft_consensus.cc:1275] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.777048 19574 consensus_queue.cc:1048] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.793370 19433 raft_consensus.cc:1275] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer ed29f8586d164987b3531a2710239e3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.795090 19574 consensus_queue.cc:1048] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.823465 19433 raft_consensus.cc:1275] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.824673 19483 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.831646 19284 raft_consensus.cc:1275] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.833313 19623 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.853305 19284 raft_consensus.cc:1275] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.854496 19623 consensus_queue.cc:1048] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.861670 19433 raft_consensus.cc:1275] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.863250 19623 consensus_queue.cc:1048] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.930254 19433 raft_consensus.cc:1275] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.931558 19623 consensus_queue.cc:1048] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.943427 19284 raft_consensus.cc:1275] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.946439 19573 consensus_queue.cc:1048] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:40.009966 19435 raft_consensus.cc:1275] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:40.011291 19573 consensus_queue.cc:1048] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:40.020540 19284 raft_consensus.cc:1275] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer c4a3067ebfa947e18a871d0ac65c2786: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:40.021906 19649 consensus_queue.cc:1048] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:40.173619 19361 raft_consensus.cc:1275] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:40.174877 19627 consensus_queue.cc:1048] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:40.183573 19286 raft_consensus.cc:1275] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:40.185315 19627 consensus_queue.cc:1048] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:42.219854 19183 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42116:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:04:42.241861 19183 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42116:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:04:42.334360  7638 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:04:42.464512 19705 tablet.cc:1724] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.471843 19702 tablet.cc:1724] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:42.493024 19688 log.cc:927] Time spent T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:04:42.499101 19692 log.cc:927] Time spent T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:04:42.505774 19387 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:f05e331d886243688815b7936d4a9394)
W20260709 13:04:42.512531 19387 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:f05e331d886243688815b7936d4a9394)
W20260709 13:04:42.516039 19387 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:f05e331d886243688815b7936d4a9394)
I20260709 13:04:42.533689 19711 tablet.cc:1724] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:42.577638 19325 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:38493, user_credentials={real_user=slave}} blocked reactor thread for 76029.5us
W20260709 13:04:42.593901 19250 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:38493, user_credentials={real_user=slave}} blocked reactor thread for 112374us
I20260709 13:04:42.601222 19717 tablet.cc:1724] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.599888 19705 tablet.cc:1724] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.621476 19712 tablet.cc:1724] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.646315 19715 tablet.cc:1724] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.619120 19702 tablet.cc:1724] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.710273 19721 tablet.cc:1724] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.714058 19720 tablet.cc:1724] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.738297 19702 tablet.cc:1724] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.741995 19726 tablet.cc:1724] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.752763 19281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
mode: GRACEFUL
new_leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:36312
I20260709 13:04:42.753881 19281 raft_consensus.cc:606] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Received request to transfer leadership to TS f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:42.772087 19725 tablet.cc:1724] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.784512 19731 tablet.cc:1724] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.787665 19719 raft_consensus.cc:993] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: : Instructing follower f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:42.788420 19719 raft_consensus.cc:1081] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Signalling peer f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:42.794931 19715 tablet.cc:1724] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.799547 19717 tablet.cc:1724] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.760507 19711 tablet.cc:1724] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.806057 19731 tablet.cc:1724] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.830519 19432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:35854
I20260709 13:04:42.831210 19432 raft_consensus.cc:493] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:42.830953 19731 tablet.cc:1724] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.831671 19432 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:42.876643 19734 tablet.cc:1724] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.875030 19432 raft_consensus.cc:515] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:42.898850 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:42.883996 19715 tablet.cc:1724] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.907272 19360 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:42.899798 19702 tablet.cc:1724] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.910118 19721 tablet.cc:1724] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.892545 19282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce6554a7e2dd4bf7a9c66e23c5dc2391" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:42.913818 19282 raft_consensus.cc:3055] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:42.914183 19282 raft_consensus.cc:740] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:42.915071 19282 consensus_queue.cc:260] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:42.916318 19282 raft_consensus.cc:3060] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:42.894779 19705 tablet.cc:1724] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.914544 19731 tablet.cc:1724] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.900323 19725 tablet.cc:1724] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.901597 19717 tablet.cc:1724] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:42.893482 19432 leader_election.cc:290] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:42.939932 19359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f05e331d886243688815b7936d4a9394"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:50140
I20260709 13:04:42.940582 19359 raft_consensus.cc:606] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Received request to transfer leadership to TS f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:42.930759 19360 raft_consensus.cc:2468] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:42.946949 19400 leader_election.cc:304] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:42.948252 19660 raft_consensus.cc:2804] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:42.948755 19660 raft_consensus.cc:697] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:42.951887 19660 consensus_queue.cc:237] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:42.956182  7638 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:04:42.956944  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:42.964619 19282 raft_consensus.cc:2468] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:42.966645 19282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e"
dest_uuid: "ed29f8586d164987b3531a2710239e3d"
mode: GRACEFUL
new_leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:36312
I20260709 13:04:42.967162 19282 raft_consensus.cc:606] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Received request to transfer leadership to TS f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:42.970237 19360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc"
dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
mode: GRACEFUL
new_leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:50140
I20260709 13:04:42.970702 19360 raft_consensus.cc:606] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Received request to transfer leadership to TS f825e4d993594c96a14fcb1a1c742ef2
I20260709 13:04:42.972009  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:42.972976  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:42.976568 19182 catalog_manager.cc:6242] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: soft_deleted_table [id=3f75217a687c42e1b5b7f985845be211] alter complete (version 1)
I20260709 13:04:42.987633 19183 catalog_manager.cc:5697] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 1 to 2, leader changed from ed29f8586d164987b3531a2710239e3d (127.7.117.129) to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:43.058699 19719 raft_consensus.cc:993] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: : Instructing follower f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.059223 19723 raft_consensus.cc:1081] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Signalling peer f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.060779 19432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:35854
I20260709 13:04:43.061326 19432 raft_consensus.cc:493] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.061676 19432 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.066881 19432 raft_consensus.cc:515] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.068755 19432 leader_election.cc:290] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:43.069603 19282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:43.069815 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0faf2a5f8d2a4600b8539dce8cbf9f9e" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:43.070307 19282 raft_consensus.cc:3055] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.070427 19360 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.070652 19282 raft_consensus.cc:740] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed29f8586d164987b3531a2710239e3d, State: Running, Role: LEADER
I20260709 13:04:43.071528 19282 consensus_queue.cc:260] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.072577 19282 raft_consensus.cc:3060] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.075841 19360 raft_consensus.cc:2468] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:43.077491 19400 leader_election.cc:304] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a3067ebfa947e18a871d0ac65c2786, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:43.078372 19660 raft_consensus.cc:2804] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:43.078886 19660 raft_consensus.cc:697] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:43.079413 19282 raft_consensus.cc:2468] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:43.080049 19660 consensus_queue.cc:237] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.088289 19182 catalog_manager.cc:5697] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 1 to 2, leader changed from ed29f8586d164987b3531a2710239e3d (127.7.117.129) to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:43.147260 19709 raft_consensus.cc:993] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.147768 19709 raft_consensus.cc:1081] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Signalling peer f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.149649 19432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:35842
I20260709 13:04:43.150256 19432 raft_consensus.cc:493] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.150523 19432 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.155408 19432 raft_consensus.cc:515] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.157127 19432 leader_election.cc:290] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:43.158324 19282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:43.158221 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b5b57968404d72bb79778c9beb66fc" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:43.159098 19282 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.159165 19360 raft_consensus.cc:3055] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.159634 19360 raft_consensus.cc:740] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:43.160470 19360 consensus_queue.cc:260] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.161871 19360 raft_consensus.cc:3060] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.166865 19282 raft_consensus.cc:2468] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:43.168169 19400 leader_election.cc:304] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:43.168877 19360 raft_consensus.cc:2468] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 2.
I20260709 13:04:43.169205 19690 raft_consensus.cc:2804] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:43.169822 19690 raft_consensus.cc:697] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:43.170819 19690 consensus_queue.cc:237] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.183591 19182 catalog_manager.cc:5697] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 1 to 2, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 2 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:43.411844 19695 raft_consensus.cc:993] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: : Instructing follower f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.412422 19709 raft_consensus.cc:1081] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Signalling peer f825e4d993594c96a14fcb1a1c742ef2 to start an election
I20260709 13:04:43.414148 19433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f05e331d886243688815b7936d4a9394"
dest_uuid: "f825e4d993594c96a14fcb1a1c742ef2"
 from {username='slave'} at 127.0.0.1:35842
I20260709 13:04:43.414705 19433 raft_consensus.cc:493] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.415030 19433 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.419490 19433 raft_consensus.cc:515] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.421031 19433 leader_election.cc:290] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 3 election: Requested vote from peers ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435), c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063)
I20260709 13:04:43.422173 19360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "c4a3067ebfa947e18a871d0ac65c2786"
I20260709 13:04:43.422187 19282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05e331d886243688815b7936d4a9394" candidate_uuid: "f825e4d993594c96a14fcb1a1c742ef2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ed29f8586d164987b3531a2710239e3d"
I20260709 13:04:43.423060 19360 raft_consensus.cc:3055] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:43.423110 19282 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.423480 19360 raft_consensus.cc:740] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4a3067ebfa947e18a871d0ac65c2786, State: Running, Role: LEADER
I20260709 13:04:43.424247 19360 consensus_queue.cc:260] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.425473 19360 raft_consensus.cc:3060] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:43.430482 19282 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 3.
I20260709 13:04:43.431921 19400 leader_election.cc:304] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed29f8586d164987b3531a2710239e3d, f825e4d993594c96a14fcb1a1c742ef2; no voters: 
I20260709 13:04:43.432310 19360 raft_consensus.cc:2468] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f825e4d993594c96a14fcb1a1c742ef2 in term 3.
I20260709 13:04:43.432860 19690 raft_consensus.cc:2804] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:43.433423 19690 raft_consensus.cc:697] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 3 LEADER]: Becoming Leader. State: Replica: f825e4d993594c96a14fcb1a1c742ef2, State: Running, Role: LEADER
I20260709 13:04:43.434191 19690 consensus_queue.cc:237] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } }
I20260709 13:04:43.440986 19182 catalog_manager.cc:5697] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 reported cstate change: term changed from 2 to 3, leader changed from c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130) to f825e4d993594c96a14fcb1a1c742ef2 (127.7.117.131). New cstate: current_term: 3 leader_uuid: "f825e4d993594c96a14fcb1a1c742ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825e4d993594c96a14fcb1a1c742ef2" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 38493 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:43.512703 19360 raft_consensus.cc:1275] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:43.514179 19690 consensus_queue.cc:1048] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:43.522581 19282 raft_consensus.cc:1275] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:43.525074 19742 consensus_queue.cc:1048] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:43.555723 19282 raft_consensus.cc:1275] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:43.557228 19742 consensus_queue.cc:1048] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.569419 19360 raft_consensus.cc:1275] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:43.570858 19690 consensus_queue.cc:1048] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.635210 19282 raft_consensus.cc:1275] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:43.636605 19690 consensus_queue.cc:1048] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.646777 19360 raft_consensus.cc:1275] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:43.648492 19742 consensus_queue.cc:1048] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.820930 19360 raft_consensus.cc:1275] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 3 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:43.822209 19742 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a3067ebfa947e18a871d0ac65c2786" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:43.831190 19282 raft_consensus.cc:1275] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 3 FOLLOWER]: Refusing update from remote peer f825e4d993594c96a14fcb1a1c742ef2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:43.832912 19746 consensus_queue.cc:1048] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed29f8586d164987b3531a2710239e3d" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 44435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:44.036027  7638 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:04:44.041055  7638 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:04:44.041471  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:44.047183  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:04:44.047525  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:45.076203  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
I20260709 13:04:45.103796  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:45.104777  7638 tablet_replica.cc:333] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.105731  7638 raft_consensus.cc:2243] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.106330  7638 raft_consensus.cc:2272] T 3c26262294aa42a1ae16a744c7e6907e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.108276  7638 tablet_replica.cc:333] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.108834  7638 raft_consensus.cc:2243] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.109328  7638 raft_consensus.cc:2272] T 570e44dbcfd141aa9aeec09ad7e06f51 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.111092  7638 tablet_replica.cc:333] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.111687  7638 raft_consensus.cc:2243] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.112057  7638 raft_consensus.cc:2272] T 0d4112f7e95442a8908cdb0100f4bf29 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.113714  7638 tablet_replica.cc:333] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.114346  7638 raft_consensus.cc:2243] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.114778  7638 raft_consensus.cc:2272] T 026310e07db848728601d2b78891d911 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.116482  7638 tablet_replica.cc:333] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.117013  7638 raft_consensus.cc:2243] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.117795  7638 raft_consensus.cc:2272] T 1716ef34dd36473896561ae661766ff4 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.119913  7638 tablet_replica.cc:333] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.120464  7638 raft_consensus.cc:2243] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.120939  7638 raft_consensus.cc:2272] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.123412  7638 tablet_replica.cc:333] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.124234  7638 raft_consensus.cc:2243] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.125068  7638 raft_consensus.cc:2272] T c8001d125aee47f3bf74213553775dfe P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.127023  7638 tablet_replica.cc:333] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.127580  7638 raft_consensus.cc:2243] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.128096  7638 raft_consensus.cc:2272] T 50b5b57968404d72bb79778c9beb66fc P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.131269  7638 tablet_replica.cc:333] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
W20260709 13:04:45.131510 19326 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
I20260709 13:04:45.132000  7638 raft_consensus.cc:2243] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.132495  7638 raft_consensus.cc:2272] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.135164  7638 tablet_replica.cc:333] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.135780  7638 raft_consensus.cc:2243] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.136240  7638 raft_consensus.cc:2272] T 48d6ee85a0fc449db61e9360433be5d3 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.138608  7638 tablet_replica.cc:333] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.139144  7638 raft_consensus.cc:2243] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.139582  7638 raft_consensus.cc:2272] T f05e331d886243688815b7936d4a9394 P ed29f8586d164987b3531a2710239e3d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.141681  7638 tablet_replica.cc:333] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.142278  7638 raft_consensus.cc:2243] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.143126  7638 raft_consensus.cc:2272] T e96e9a71181b4523b82e78bc0e093f5e P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:45.144335 19326 consensus_peers.cc:597] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.145733  7638 tablet_replica.cc:333] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.146364  7638 raft_consensus.cc:2243] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.147266  7638 raft_consensus.cc:2272] T 321f8b0faf5b4e4a88fab2da1e36f384 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.150136  7638 tablet_replica.cc:333] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.150916  7638 raft_consensus.cc:2243] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.151420  7638 raft_consensus.cc:2272] T 86e4e8a1c6994730bc90ae084a99faad P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.153530  7638 tablet_replica.cc:333] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.154047  7638 raft_consensus.cc:2243] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.154445  7638 raft_consensus.cc:2272] T 153e8097ed504596872f548f628f6d71 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.157281  7638 tablet_replica.cc:333] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.157898  7638 raft_consensus.cc:2243] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.158836  7638 raft_consensus.cc:2272] T cbcc52ac04e444c4b343c7794b8db3bb P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.160952  7638 tablet_replica.cc:333] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.161505  7638 raft_consensus.cc:2243] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.162215  7638 raft_consensus.cc:2272] T 478d1bf765e44df48a5fe76d66666df2 P ed29f8586d164987b3531a2710239e3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.164041  7638 tablet_replica.cc:333] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d: stopping tablet replica
I20260709 13:04:45.164477  7638 raft_consensus.cc:2243] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.164887  7638 raft_consensus.cc:2272] T dd76380481434977ab62d183db59a9d2 P ed29f8586d164987b3531a2710239e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.203670  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
W20260709 13:04:45.238077 19326 consensus_peers.cc:597] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:44435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.240716  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
I20260709 13:04:45.275231  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:45.275897  7638 tablet_replica.cc:333] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.276511  7638 raft_consensus.cc:2243] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.277017  7638 raft_consensus.cc:2272] T 3c26262294aa42a1ae16a744c7e6907e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.278905  7638 tablet_replica.cc:333] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.279448  7638 raft_consensus.cc:2243] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.279870  7638 raft_consensus.cc:2272] T f05e331d886243688815b7936d4a9394 P c4a3067ebfa947e18a871d0ac65c2786 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.281891  7638 tablet_replica.cc:333] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.282471  7638 raft_consensus.cc:2243] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.282912  7638 raft_consensus.cc:2272] T c8001d125aee47f3bf74213553775dfe P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.284922  7638 tablet_replica.cc:333] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.285498  7638 raft_consensus.cc:2243] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.286209  7638 raft_consensus.cc:2272] T 570e44dbcfd141aa9aeec09ad7e06f51 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.287842  7638 tablet_replica.cc:333] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.288390  7638 raft_consensus.cc:2243] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.289098  7638 raft_consensus.cc:2272] T 0d4112f7e95442a8908cdb0100f4bf29 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.291002  7638 tablet_replica.cc:333] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.291567  7638 raft_consensus.cc:2243] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.292032  7638 raft_consensus.cc:2272] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.293918  7638 tablet_replica.cc:333] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.294462  7638 raft_consensus.cc:2243] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.295306  7638 raft_consensus.cc:2272] T 026310e07db848728601d2b78891d911 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.297211  7638 tablet_replica.cc:333] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.297818  7638 raft_consensus.cc:2243] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.298278  7638 raft_consensus.cc:2272] T 1716ef34dd36473896561ae661766ff4 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.300133  7638 tablet_replica.cc:333] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.300640  7638 raft_consensus.cc:2243] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.301043  7638 raft_consensus.cc:2272] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.302968  7638 tablet_replica.cc:333] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.303495  7638 raft_consensus.cc:2243] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.303994  7638 raft_consensus.cc:2272] T e96e9a71181b4523b82e78bc0e093f5e P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.306178  7638 tablet_replica.cc:333] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.306842  7638 raft_consensus.cc:2243] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.307462  7638 raft_consensus.cc:2272] T 86e4e8a1c6994730bc90ae084a99faad P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.310407  7638 tablet_replica.cc:333] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.311144  7638 raft_consensus.cc:2243] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.311652  7638 raft_consensus.cc:2272] T 321f8b0faf5b4e4a88fab2da1e36f384 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:45.312309 19400 consensus_peers.cc:597] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:44435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.314155  7638 tablet_replica.cc:333] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.314746  7638 raft_consensus.cc:2243] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.315166  7638 raft_consensus.cc:2272] T cbcc52ac04e444c4b343c7794b8db3bb P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.317116  7638 tablet_replica.cc:333] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.317838  7638 raft_consensus.cc:2243] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.318686  7638 raft_consensus.cc:2272] T 153e8097ed504596872f548f628f6d71 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.320644  7638 tablet_replica.cc:333] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.321177  7638 raft_consensus.cc:2243] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.321642  7638 raft_consensus.cc:2272] T 478d1bf765e44df48a5fe76d66666df2 P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.323539  7638 tablet_replica.cc:333] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.324054  7638 raft_consensus.cc:2243] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.324816  7638 raft_consensus.cc:2272] T dd76380481434977ab62d183db59a9d2 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.326725  7638 tablet_replica.cc:333] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.327195  7638 raft_consensus.cc:2243] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.327935  7638 raft_consensus.cc:2272] T 48d6ee85a0fc449db61e9360433be5d3 P c4a3067ebfa947e18a871d0ac65c2786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.329790  7638 tablet_replica.cc:333] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786: stopping tablet replica
I20260709 13:04:45.330252  7638 raft_consensus.cc:2243] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.330695  7638 raft_consensus.cc:2272] T 50b5b57968404d72bb79778c9beb66fc P c4a3067ebfa947e18a871d0ac65c2786 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:45.365739 19400 consensus_peers.cc:597] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:44435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.369912  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
I20260709 13:04:45.402400  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
W20260709 13:04:45.417891 19400 consensus_peers.cc:597] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 -> Peer c4a3067ebfa947e18a871d0ac65c2786 (127.7.117.130:38063): Couldn't send request to peer c4a3067ebfa947e18a871d0ac65c2786. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.130:38063: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:45.428654 19400 consensus_peers.cc:597] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:44435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.441385  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:45.442308  7638 tablet_replica.cc:333] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.442974  7638 raft_consensus.cc:2243] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.443970  7638 raft_consensus.cc:2272] T 3c26262294aa42a1ae16a744c7e6907e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.446295  7638 tablet_replica.cc:333] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.447074  7638 raft_consensus.cc:2243] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:45.447235 19400 consensus_peers.cc:597] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 -> Peer ed29f8586d164987b3531a2710239e3d (127.7.117.129:44435): Couldn't send request to peer ed29f8586d164987b3531a2710239e3d. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:44435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.447574  7638 raft_consensus.cc:2272] T 570e44dbcfd141aa9aeec09ad7e06f51 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.449594  7638 tablet_replica.cc:333] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.450174  7638 raft_consensus.cc:2243] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.450583  7638 raft_consensus.cc:2272] T 0d4112f7e95442a8908cdb0100f4bf29 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.452272  7638 tablet_replica.cc:333] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.452822  7638 raft_consensus.cc:2243] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.453744  7638 raft_consensus.cc:2272] T ce6554a7e2dd4bf7a9c66e23c5dc2391 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.455566  7638 tablet_replica.cc:333] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.456105  7638 raft_consensus.cc:2243] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.456534  7638 raft_consensus.cc:2272] T 026310e07db848728601d2b78891d911 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.458215  7638 tablet_replica.cc:333] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.458750  7638 raft_consensus.cc:2243] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.459168  7638 raft_consensus.cc:2272] T 1716ef34dd36473896561ae661766ff4 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.460839  7638 tablet_replica.cc:333] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.461417  7638 raft_consensus.cc:2243] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.462136  7638 raft_consensus.cc:2272] T f05e331d886243688815b7936d4a9394 P f825e4d993594c96a14fcb1a1c742ef2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.464037  7638 tablet_replica.cc:333] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.464584  7638 raft_consensus.cc:2243] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.464988  7638 raft_consensus.cc:2272] T c8001d125aee47f3bf74213553775dfe P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.466764  7638 tablet_replica.cc:333] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.467267  7638 raft_consensus.cc:2243] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.467998  7638 raft_consensus.cc:2272] T 0faf2a5f8d2a4600b8539dce8cbf9f9e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.469861  7638 tablet_replica.cc:333] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.470371  7638 raft_consensus.cc:2243] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.471100  7638 raft_consensus.cc:2272] T 86e4e8a1c6994730bc90ae084a99faad P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.473124  7638 tablet_replica.cc:333] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.473654  7638 raft_consensus.cc:2243] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.474066  7638 raft_consensus.cc:2272] T 321f8b0faf5b4e4a88fab2da1e36f384 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.475824  7638 tablet_replica.cc:333] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.476322  7638 raft_consensus.cc:2243] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.476739  7638 raft_consensus.cc:2272] T e96e9a71181b4523b82e78bc0e093f5e P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.478659  7638 tablet_replica.cc:333] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.479152  7638 raft_consensus.cc:2243] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.479568  7638 raft_consensus.cc:2272] T cbcc52ac04e444c4b343c7794b8db3bb P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.481392  7638 tablet_replica.cc:333] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.481884  7638 raft_consensus.cc:2243] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.482295  7638 raft_consensus.cc:2272] T 153e8097ed504596872f548f628f6d71 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.484123  7638 tablet_replica.cc:333] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.484670  7638 raft_consensus.cc:2243] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.485095  7638 raft_consensus.cc:2272] T 478d1bf765e44df48a5fe76d66666df2 P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.486881  7638 tablet_replica.cc:333] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.487385  7638 raft_consensus.cc:2243] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.487787  7638 raft_consensus.cc:2272] T dd76380481434977ab62d183db59a9d2 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.489656  7638 tablet_replica.cc:333] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.490115  7638 raft_consensus.cc:2243] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.490526  7638 raft_consensus.cc:2272] T 48d6ee85a0fc449db61e9360433be5d3 P f825e4d993594c96a14fcb1a1c742ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.492434  7638 tablet_replica.cc:333] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2: stopping tablet replica
I20260709 13:04:45.492903  7638 raft_consensus.cc:2243] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.493782  7638 raft_consensus.cc:2272] T 50b5b57968404d72bb79778c9beb66fc P f825e4d993594c96a14fcb1a1c742ef2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.532294  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 13:04:45.565209  7638 master.cc:562] Master@127.7.117.190:39289 shutting down...
I20260709 13:04:45.588003  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:45.588619  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.588980  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P d76cb895aa1246a18bbd4f78f4a10d6e: stopping tablet replica
I20260709 13:04:45.610572  7638 master.cc:584] Master@127.7.117.190:39289 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13081 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:04:45.669602  7638 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.117.190:42699
I20260709 13:04:45.670661  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:45.676466 19759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:45.676501 19758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:45.676785 19761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:45.678228  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:45.679039  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:45.679211  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:45.679327  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602285679315 us; error 0 us; skew 500 ppm
I20260709 13:04:45.679754  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:45.682209  7638 webserver.cc:533] Webserver started at http://127.7.117.190:36051/ using document root <none> and password file <none>
I20260709 13:04:45.682619  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:45.682770  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:45.682982  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:45.683987  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/master-0-root/instance:
uuid: "de635c90a3884ef79a0dc62fe61f6008"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.688230  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:04:45.691465 19766 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:45.692219  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:04:45.692508  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/master-0-root
uuid: "de635c90a3884ef79a0dc62fe61f6008"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.692796  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:45.716457  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:45.717671  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:45.756690  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.190:42699
I20260709 13:04:45.756808 19817 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.190:42699 every 8 connection(s)
I20260709 13:04:45.760509 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.771786 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: Bootstrap starting.
I20260709 13:04:45.776953 19818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.781613 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: No bootstrap required, opened a new log
I20260709 13:04:45.783794 19818 raft_consensus.cc:359] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER }
I20260709 13:04:45.784166 19818 raft_consensus.cc:385] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.784376 19818 raft_consensus.cc:740] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de635c90a3884ef79a0dc62fe61f6008, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.785043 19818 consensus_queue.cc:260] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER }
I20260709 13:04:45.785532 19818 raft_consensus.cc:399] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:45.785733 19818 raft_consensus.cc:493] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:45.785940 19818 raft_consensus.cc:3060] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.791501 19818 raft_consensus.cc:515] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER }
I20260709 13:04:45.792004 19818 leader_election.cc:304] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de635c90a3884ef79a0dc62fe61f6008; no voters: 
I20260709 13:04:45.793059 19818 leader_election.cc:290] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:45.793440 19821 raft_consensus.cc:2804] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.794885 19821 raft_consensus.cc:697] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 1 LEADER]: Becoming Leader. State: Replica: de635c90a3884ef79a0dc62fe61f6008, State: Running, Role: LEADER
I20260709 13:04:45.795594 19821 consensus_queue.cc:237] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER }
I20260709 13:04:45.796263 19818 sys_catalog.cc:565] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:45.800240 19822 sys_catalog.cc:455] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de635c90a3884ef79a0dc62fe61f6008" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER } }
I20260709 13:04:45.801034 19822 sys_catalog.cc:458] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:45.802294 19823 sys_catalog.cc:455] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de635c90a3884ef79a0dc62fe61f6008. Latest consensus state: current_term: 1 leader_uuid: "de635c90a3884ef79a0dc62fe61f6008" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de635c90a3884ef79a0dc62fe61f6008" member_type: VOTER } }
I20260709 13:04:45.802968 19823 sys_catalog.cc:458] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:45.803642 19827 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:45.812006 19827 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:45.813025  7638 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:45.820792 19827 catalog_manager.cc:1383] Generated new cluster ID: e45becdb9c454c58a0879898ebb81ee0
I20260709 13:04:45.821094 19827 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:45.842455 19827 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:45.843796 19827 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:45.869679 19827 catalog_manager.cc:6070] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: Generated new TSK 0
I20260709 13:04:45.870306 19827 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:45.879881  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:45.885823 19839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:45.886759 19840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:45.888866  7638 server_base.cc:1061] running on GCE node
W20260709 13:04:45.889144 19842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:45.890277  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:45.890470  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:45.890707  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602285890687 us; error 0 us; skew 500 ppm
I20260709 13:04:45.891224  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:45.893632  7638 webserver.cc:533] Webserver started at http://127.7.117.129:38791/ using document root <none> and password file <none>
I20260709 13:04:45.894097  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:45.894311  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:45.894693  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:45.895757  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-0-root/instance:
uuid: "aa705cbdb57641be99f49d5e3c4225e6"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.900267  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:04:45.903558 19847 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:45.904378  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:45.904714  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-0-root
uuid: "aa705cbdb57641be99f49d5e3c4225e6"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.904969  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:45.917815  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:45.919006  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:45.920434  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:45.922683  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:45.922856  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:45.923156  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:45.923309  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:45.963573  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.129:34441
I20260709 13:04:45.963708 19909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.129:34441 every 8 connection(s)
I20260709 13:04:45.968742  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:45.976069 19914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:45.977949 19915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:45.980216  7638 server_base.cc:1061] running on GCE node
W20260709 13:04:45.980261 19917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:45.981633  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:45.981920  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:45.982126  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602285982087 us; error 0 us; skew 500 ppm
I20260709 13:04:45.982825  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:45.983145 19910 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42699
I20260709 13:04:45.983616 19910 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:45.984550 19910 heartbeater.cc:507] Master 127.7.117.190:42699 requested a full tablet report, sending...
I20260709 13:04:45.985893  7638 webserver.cc:533] Webserver started at http://127.7.117.130:43311/ using document root <none> and password file <none>
I20260709 13:04:45.986474  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:45.986706  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:45.987021  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:45.987200 19783 ts_manager.cc:194] Registered new tserver with Master: aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441)
I20260709 13:04:45.988523  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-1-root/instance:
uuid: "79b099cf2a294ddea68376a11141032f"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.989661 19783 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36790
I20260709 13:04:45.994190  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:04:45.997857 19922 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:45.998652  7638 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:04:45.998943  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-1-root
uuid: "79b099cf2a294ddea68376a11141032f"
format_stamp: "Formatted at 2026-07-09 13:04:45 on dist-test-slave-ndzg"
I20260709 13:04:45.999225  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:46.011519  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:46.012715  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:46.014222  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:46.016404  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:46.016636  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:46.016912  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:46.017071  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:46.056037  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.130:39275
I20260709 13:04:46.056129 19984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.130:39275 every 8 connection(s)
I20260709 13:04:46.060899  7638 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:46.068919 19988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:46.068954 19989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:46.071872 19991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:46.072389 19985 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42699
I20260709 13:04:46.072829 19985 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:46.073211  7638 server_base.cc:1061] running on GCE node
I20260709 13:04:46.073751 19985 heartbeater.cc:507] Master 127.7.117.190:42699 requested a full tablet report, sending...
I20260709 13:04:46.074302  7638 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:46.074568  7638 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:46.074759  7638 hybrid_clock.cc:648] HybridClock initialized: now 1783602286074724 us; error 0 us; skew 500 ppm
I20260709 13:04:46.075486  7638 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:46.075925 19783 ts_manager.cc:194] Registered new tserver with Master: 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:46.077915 19783 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36792
I20260709 13:04:46.078351  7638 webserver.cc:533] Webserver started at http://127.7.117.131:46621/ using document root <none> and password file <none>
I20260709 13:04:46.078977  7638 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:46.079211  7638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:46.079494  7638 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:46.080672  7638 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-2-root/instance:
uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
format_stamp: "Formatted at 2026-07-09 13:04:46 on dist-test-slave-ndzg"
I20260709 13:04:46.084972  7638 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:04:46.089200 19996 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:46.090083  7638 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:46.090389  7638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-2-root
uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
format_stamp: "Formatted at 2026-07-09 13:04:46 on dist-test-slave-ndzg"
I20260709 13:04:46.090694  7638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task8rXhnP/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935877741-7638-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:46.126364  7638 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:46.127529  7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:46.128947  7638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:46.131279  7638 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:46.131518  7638 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:46.131759  7638 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:46.131934  7638 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:46.172609  7638 rpc_server.cc:307] RPC server started. Bound to: 127.7.117.131:35871
I20260709 13:04:46.172714 20058 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.117.131:35871 every 8 connection(s)
I20260709 13:04:46.186199 20059 heartbeater.cc:344] Connected to a master server at 127.7.117.190:42699
I20260709 13:04:46.186602 20059 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:46.187287 20059 heartbeater.cc:507] Master 127.7.117.190:42699 requested a full tablet report, sending...
I20260709 13:04:46.189152 19783 ts_manager.cc:194] Registered new tserver with Master: 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.189397  7638 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013590386s
I20260709 13:04:46.190596 19783 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36798
I20260709 13:04:46.219843 19783 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36806:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:46.222577 19783 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:46.306162 19949 tablet_service.cc:1511] Processing CreateTablet for tablet d601750e621f4e9a83d1153e5b946a7e (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:46.307574 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d601750e621f4e9a83d1153e5b946a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.308135 20024 tablet_service.cc:1511] Processing CreateTablet for tablet 2d08efba25a74cb69708233eef807713 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:46.308878 20023 tablet_service.cc:1511] Processing CreateTablet for tablet d601750e621f4e9a83d1153e5b946a7e (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:46.309566 20024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d08efba25a74cb69708233eef807713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.310266 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d601750e621f4e9a83d1153e5b946a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.310382 19948 tablet_service.cc:1511] Processing CreateTablet for tablet 5041e1d57a3c4ec88074674cabbdf7ef (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:46.311780 19948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041e1d57a3c4ec88074674cabbdf7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.312282 19874 tablet_service.cc:1511] Processing CreateTablet for tablet d601750e621f4e9a83d1153e5b946a7e (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:46.312546 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 5041e1d57a3c4ec88074674cabbdf7ef (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:46.313681 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d601750e621f4e9a83d1153e5b946a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.316679 19875 tablet_service.cc:1511] Processing CreateTablet for tablet 2d08efba25a74cb69708233eef807713 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:46.317426 19950 tablet_service.cc:1511] Processing CreateTablet for tablet 2d08efba25a74cb69708233eef807713 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:46.318638 19950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d08efba25a74cb69708233eef807713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.316601 19872 tablet_service.cc:1511] Processing CreateTablet for tablet 45de4865bd314c65b42a7121fdd2720b (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:46.318986 19947 tablet_service.cc:1511] Processing CreateTablet for tablet 45de4865bd314c65b42a7121fdd2720b (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:46.319633 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041e1d57a3c4ec88074674cabbdf7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.320322 19947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45de4865bd314c65b42a7121fdd2720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.321496 19875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d08efba25a74cb69708233eef807713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.318202 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 5041e1d57a3c4ec88074674cabbdf7ef (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:46.323051 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041e1d57a3c4ec88074674cabbdf7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.329671 19872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45de4865bd314c65b42a7121fdd2720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.369407 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 45de4865bd314c65b42a7121fdd2720b (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:46.370847 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45de4865bd314c65b42a7121fdd2720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.384248 19950 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8e89b7ec540cb9eda24fd98ef7acc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:46.386051 19950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8e89b7ec540cb9eda24fd98ef7acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.386054 20020 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8e89b7ec540cb9eda24fd98ef7acc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:46.390018 20020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8e89b7ec540cb9eda24fd98ef7acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.400138 19875 tablet_service.cc:1511] Processing CreateTablet for tablet 28c8e89b7ec540cb9eda24fd98ef7acc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:46.400727 19873 tablet_service.cc:1511] Processing CreateTablet for tablet dfbbbe561e014873b8e6fe086f3a8b31 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:46.401635 19875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c8e89b7ec540cb9eda24fd98ef7acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.402287 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbbbe561e014873b8e6fe086f3a8b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.405764 19946 tablet_service.cc:1511] Processing CreateTablet for tablet dfbbbe561e014873b8e6fe086f3a8b31 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:46.409727 19946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbbbe561e014873b8e6fe086f3a8b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.409834 20019 tablet_service.cc:1511] Processing CreateTablet for tablet dfbbbe561e014873b8e6fe086f3a8b31 (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:46.411017 20019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbbbe561e014873b8e6fe086f3a8b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.417989 20075 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.427496 20076 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.427815 20074 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.437664 20075 tablet_bootstrap.cc:654] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.441728 20074 tablet_bootstrap.cc:654] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.443645 19872 tablet_service.cc:1511] Processing CreateTablet for tablet 0b0cf2c1fe954feba34a9cb23532cbca (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:46.452441 20024 tablet_service.cc:1511] Processing CreateTablet for tablet 713e842bd00941b4b931176299bc5ada (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:46.446410 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 0b0cf2c1fe954feba34a9cb23532cbca (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:46.448060 19871 tablet_service.cc:1511] Processing CreateTablet for tablet 713e842bd00941b4b931176299bc5ada (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:46.454344 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0cf2c1fe954feba34a9cb23532cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.454371 19872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0cf2c1fe954feba34a9cb23532cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.460145 19871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713e842bd00941b4b931176299bc5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.461796 20024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713e842bd00941b4b931176299bc5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.452346 19950 tablet_service.cc:1511] Processing CreateTablet for tablet 713e842bd00941b4b931176299bc5ada (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:46.468683 19950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713e842bd00941b4b931176299bc5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.469477 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 153d1a15c28f47b8b822a41ab43d1adc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:46.470736 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153d1a15c28f47b8b822a41ab43d1adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.462891 20076 tablet_bootstrap.cc:654] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.450502 19946 tablet_service.cc:1511] Processing CreateTablet for tablet 0b0cf2c1fe954feba34a9cb23532cbca (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:46.478570 19946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0cf2c1fe954feba34a9cb23532cbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.480926 19949 tablet_service.cc:1511] Processing CreateTablet for tablet 153d1a15c28f47b8b822a41ab43d1adc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:46.482779 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153d1a15c28f47b8b822a41ab43d1adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.488847 20019 tablet_service.cc:1511] Processing CreateTablet for tablet 153d1a15c28f47b8b822a41ab43d1adc (DEFAULT_TABLE table=test-workload [id=7731bbc9068e4d8ca58cf8977bfb6749]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:46.490192 20019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153d1a15c28f47b8b822a41ab43d1adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:46.491600 20076 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:46.492213 20076 ts_tablet_manager.cc:1403] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.000s
I20260709 13:04:46.496032 20076 raft_consensus.cc:359] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.498860 20075 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.507664 20076 raft_consensus.cc:385] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.507987 20075 ts_tablet_manager.cc:1403] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.090s	user 0.019s	sys 0.016s
I20260709 13:04:46.508108 20076 raft_consensus.cc:740] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.511826 20075 raft_consensus.cc:359] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.512564 20075 raft_consensus.cc:385] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.512871 20075 raft_consensus.cc:740] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.513742 20076 consensus_queue.cc:260] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.513923 20075 consensus_queue.cc:260] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.521152 20074 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.521986 20074 ts_tablet_manager.cc:1403] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.094s	user 0.016s	sys 0.000s
I20260709 13:04:46.534574 20059 heartbeater.cc:499] Master 127.7.117.190:42699 was elected leader, sending a full tablet report...
I20260709 13:04:46.535220 20076 ts_tablet_manager.cc:1434] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.043s	user 0.012s	sys 0.013s
I20260709 13:04:46.536348 20076 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.542932 20076 tablet_bootstrap.cc:654] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.548506 19910 heartbeater.cc:499] Master 127.7.117.190:42699 was elected leader, sending a full tablet report...
I20260709 13:04:46.549001 20075 ts_tablet_manager.cc:1434] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.041s	user 0.015s	sys 0.023s
I20260709 13:04:46.549114 20074 raft_consensus.cc:359] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.550209 20074 raft_consensus.cc:385] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.550285 20075 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.550624 20074 raft_consensus.cc:740] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.551726 20074 consensus_queue.cc:260] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.557006 20075 tablet_bootstrap.cc:654] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.558096 19985 heartbeater.cc:499] Master 127.7.117.190:42699 was elected leader, sending a full tablet report...
I20260709 13:04:46.558709 20074 ts_tablet_manager.cc:1434] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.004s
I20260709 13:04:46.559808 20074 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
W20260709 13:04:46.565146 19986 tablet.cc:2406] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:46.566082 20074 tablet_bootstrap.cc:654] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.569432 20076 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:46.569801 20075 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.570011 20076 ts_tablet_manager.cc:1403] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:04:46.570353 20075 ts_tablet_manager.cc:1403] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.005s
I20260709 13:04:46.573372 20076 raft_consensus.cc:359] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.574167 20076 raft_consensus.cc:385] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.573820 20075 raft_consensus.cc:359] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.574589 20076 raft_consensus.cc:740] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.574914 20075 raft_consensus.cc:385] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.575397 20075 raft_consensus.cc:740] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.575686 20076 consensus_queue.cc:260] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.576581 20075 consensus_queue.cc:260] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.580726 20075 ts_tablet_manager.cc:1434] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:04:46.582036 20075 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.583238 20076 ts_tablet_manager.cc:1434] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:04:46.584429 20076 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.590333 20075 tablet_bootstrap.cc:654] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.590736 20076 tablet_bootstrap.cc:654] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.603022 20075 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.603533 20075 ts_tablet_manager.cc:1403] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 13:04:46.604220 20074 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.604789 20074 ts_tablet_manager.cc:1403] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.015s
I20260709 13:04:46.606447 20075 raft_consensus.cc:359] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.607319 20075 raft_consensus.cc:385] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.607713 20075 raft_consensus.cc:740] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.608445 20074 raft_consensus.cc:359] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.608563 20075 consensus_queue.cc:260] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.609395 20074 raft_consensus.cc:385] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.609804 20074 raft_consensus.cc:740] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.610590 20074 consensus_queue.cc:260] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.611083 20076 tablet_bootstrap.cc:492] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:46.611701 20076 ts_tablet_manager.cc:1403] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.009s
I20260709 13:04:46.611851 20075 ts_tablet_manager.cc:1434] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:46.612948 20075 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.614950 20074 ts_tablet_manager.cc:1434] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:04:46.614904 20076 raft_consensus.cc:359] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.615818 20076 raft_consensus.cc:385] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.616055 20074 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.616204 20076 raft_consensus.cc:740] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.617158 20076 consensus_queue.cc:260] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.618940 20075 tablet_bootstrap.cc:654] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.622985 20074 tablet_bootstrap.cc:654] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.630034 20076 ts_tablet_manager.cc:1434] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:04:46.635958 20076 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.643945 20076 tablet_bootstrap.cc:654] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.647205 20074 tablet_bootstrap.cc:492] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.647627 20075 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.647826 20074 ts_tablet_manager.cc:1403] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.016s
I20260709 13:04:46.648206 20075 ts_tablet_manager.cc:1403] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.010s
I20260709 13:04:46.649166 20076 tablet_bootstrap.cc:492] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:46.649716 20076 ts_tablet_manager.cc:1403] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.008s
I20260709 13:04:46.650872 20074 raft_consensus.cc:359] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.651083 20075 raft_consensus.cc:359] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.651690 20074 raft_consensus.cc:385] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.651820 20075 raft_consensus.cc:385] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.652079 20074 raft_consensus.cc:740] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.652184 20075 raft_consensus.cc:740] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.651966 20076 raft_consensus.cc:359] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.653065 20076 raft_consensus.cc:385] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.653427 20076 raft_consensus.cc:740] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.653214 20074 consensus_queue.cc:260] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.653360 20075 consensus_queue.cc:260] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.654383 20076 consensus_queue.cc:260] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.655746 20075 ts_tablet_manager.cc:1434] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:46.656730 20075 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.655764 20074 ts_tablet_manager.cc:1434] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:46.658324 20074 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.659435 20076 ts_tablet_manager.cc:1434] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:04:46.660665 20076 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.663228 20075 tablet_bootstrap.cc:654] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.664506 20074 tablet_bootstrap.cc:654] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.667682 20076 tablet_bootstrap.cc:654] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.689899 20081 raft_consensus.cc:493] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.690708 20081 raft_consensus.cc:515] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.701553 20081 leader_election.cc:290] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.729136 20075 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.729585 20074 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.729854 20075 ts_tablet_manager.cc:1403] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.073s	user 0.030s	sys 0.039s
I20260709 13:04:46.730144 20074 ts_tablet_manager.cc:1403] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.072s	user 0.026s	sys 0.044s
I20260709 13:04:46.731681 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041e1d57a3c4ec88074674cabbdf7ef" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:46.732427 20034 raft_consensus.cc:2468] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.733129 20075 raft_consensus.cc:359] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.733577 20081 raft_consensus.cc:493] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.733497 19960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041e1d57a3c4ec88074674cabbdf7ef" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.733897 20075 raft_consensus.cc:385] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.734326 20075 raft_consensus.cc:740] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.734351 19960 raft_consensus.cc:2468] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.734359 20074 raft_consensus.cc:359] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.734234 20081 raft_consensus.cc:515] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.735610 20074 raft_consensus.cc:385] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.735512 20075 consensus_queue.cc:260] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.736160 20074 raft_consensus.cc:740] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.737257 20081 leader_election.cc:290] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.735080 19850 leader_election.cc:304] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.737161 20074 consensus_queue.cc:260] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.738597 20075 ts_tablet_manager.cc:1434] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:04:46.739642 20075 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.739682 20074 ts_tablet_manager.cc:1434] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:04:46.740999 20074 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.742177 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:46.742848 20034 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.744153 19850 leader_election.cc:304] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.746259 19960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.746943 20075 tablet_bootstrap.cc:654] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.747611 20074 tablet_bootstrap.cc:654] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.746975 19960 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.751992 20096 raft_consensus.cc:2804] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.752420 20096 raft_consensus.cc:493] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.760684 20076 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:46.752985 20081 raft_consensus.cc:2804] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.765148 20080 raft_consensus.cc:493] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.765692 20076 ts_tablet_manager.cc:1403] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.105s	user 0.024s	sys 0.027s
I20260709 13:04:46.752767 20096 raft_consensus.cc:3060] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.765774 20080 raft_consensus.cc:515] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.765616 20081 raft_consensus.cc:493] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.769093 20076 raft_consensus.cc:359] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.770012 20081 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.771653 20076 raft_consensus.cc:385] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.773738 20076 raft_consensus.cc:740] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.774834 20076 consensus_queue.cc:260] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.779109 20074 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.779999 20074 ts_tablet_manager.cc:1403] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.008s
I20260709 13:04:46.785059 20080 leader_election.cc:290] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:46.792423 20074 raft_consensus.cc:359] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.793249 20074 raft_consensus.cc:385] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.793583 20074 raft_consensus.cc:740] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.793790 20076 ts_tablet_manager.cc:1434] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.005s
I20260709 13:04:46.794365 20074 consensus_queue.cc:260] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.795253 20076 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.796663 20074 ts_tablet_manager.cc:1434] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.002s
I20260709 13:04:46.801731 20074 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.803895 20081 raft_consensus.cc:515] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.805124 20096 raft_consensus.cc:515] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
W20260709 13:04:46.808471 19831 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:46.808838 19960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:46.809757 19960 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.814538 20081 leader_election.cc:290] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.814827 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:46.819603 20034 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.829005 20101 raft_consensus.cc:493] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.829856 20101 raft_consensus.cc:515] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.835741 20075 tablet_bootstrap.cc:492] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.836410 20075 ts_tablet_manager.cc:1403] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.097s	user 0.027s	sys 0.024s
I20260709 13:04:46.836835 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041e1d57a3c4ec88074674cabbdf7ef" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:46.837560 19959 raft_consensus.cc:3060] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.839759 20075 raft_consensus.cc:359] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.840570 20075 raft_consensus.cc:385] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.840868 20075 raft_consensus.cc:740] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.845331 19959 raft_consensus.cc:2468] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:46.845597 20074 tablet_bootstrap.cc:654] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.846375 20101 leader_election.cc:290] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441)
I20260709 13:04:46.846833 19851 leader_election.cc:304] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.847919 20083 raft_consensus.cc:493] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.848423 20083 raft_consensus.cc:515] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.849052 20075 consensus_queue.cc:260] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.850530 20097 raft_consensus.cc:2804] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.856353 20076 tablet_bootstrap.cc:654] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.857774 20075 ts_tablet_manager.cc:1434] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 13:04:46.859313 20083 leader_election.cc:290] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.860754 19960 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:46.863145 20097 raft_consensus.cc:697] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:46.864112 20096 leader_election.cc:290] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.865005 20075 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.864941 19851 leader_election.cc:304] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.866230 20104 raft_consensus.cc:2804] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.869141 20104 raft_consensus.cc:697] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:46.869729 20097 consensus_queue.cc:237] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.866746 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.867605 20101 raft_consensus.cc:493] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.862080 20074 tablet_bootstrap.cc:492] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.872344 20101 raft_consensus.cc:515] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.872675 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041e1d57a3c4ec88074674cabbdf7ef" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:46.873006 20074 ts_tablet_manager.cc:1403] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.071s	user 0.005s	sys 0.006s
I20260709 13:04:46.873368 20033 raft_consensus.cc:3060] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.870085 20104 consensus_queue.cc:237] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.875658 20034 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:46.877687 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.877767 20074 raft_consensus.cc:359] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.878366 19959 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:46.878592 20074 raft_consensus.cc:385] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.878981 20074 raft_consensus.cc:740] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.880635 20076 tablet_bootstrap.cc:492] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
W20260709 13:04:46.880779 20000 leader_election.cc:343] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:46.881353 20076 ts_tablet_manager.cc:1403] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.005s
I20260709 13:04:46.881477 20033 raft_consensus.cc:2468] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:46.881604 20000 leader_election.cc:304] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:46.884392 19885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:46.885144 19885 raft_consensus.cc:2468] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:46.884984 20074 consensus_queue.cc:260] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.885691 19883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:46.886359 19883 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:46.885131 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:46.887267 19998 leader_election.cc:304] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 79b099cf2a294ddea68376a11141032f
I20260709 13:04:46.888027 19884 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:46.888671 19998 leader_election.cc:304] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.888859 20080 raft_consensus.cc:2804] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.889444 20080 raft_consensus.cc:493] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.883363 20075 tablet_bootstrap.cc:654] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.889833 20080 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.890908 20076 raft_consensus.cc:359] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.891741 20076 raft_consensus.cc:385] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.892097 20076 raft_consensus.cc:740] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.887405 20074 ts_tablet_manager.cc:1434] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:04:46.892917 20076 consensus_queue.cc:260] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.894450 20074 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:46.900033 20080 raft_consensus.cc:515] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.903729 20080 leader_election.cc:290] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:46.904289 20080 raft_consensus.cc:2804] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.904644 20080 raft_consensus.cc:493] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.904982 20080 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.907896 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:46.909762 20101 leader_election.cc:290] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:46.910753 20101 raft_consensus.cc:2804] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.911109 20101 raft_consensus.cc:493] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.911437 20101 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.916374 20074 tablet_bootstrap.cc:654] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.916639 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:46.917445 19959 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.923909 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.924808 19958 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:46.938457 20075 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:46.947479 20075 ts_tablet_manager.cc:1403] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.015s
I20260709 13:04:46.938457 20076 ts_tablet_manager.cc:1434] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.057s	user 0.012s	sys 0.016s
I20260709 13:04:46.944304 20096 raft_consensus.cc:493] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.949913 20096 raft_consensus.cc:515] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.946213 19783 catalog_manager.cc:5697] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.950981 19959 raft_consensus.cc:2468] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:46.952162 20096 leader_election.cc:290] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:46.953019 20076 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:46.952843 20101 raft_consensus.cc:515] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.954640 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:46.955041 20101 leader_election.cc:290] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441)
I20260709 13:04:46.955343 20034 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.956800 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:46.957463 19958 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.957911 20000 leader_election.cc:304] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
W20260709 13:04:46.940307 19998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.117.129:34441, user_credentials={real_user=slave}} blocked reactor thread for 51708us
I20260709 13:04:46.949151 20074 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:46.960008 20074 ts_tablet_manager.cc:1403] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.000s
I20260709 13:04:46.960608 20075 raft_consensus.cc:359] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.961596 19883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:46.944891 20080 raft_consensus.cc:515] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.962214 19883 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.962736 20101 raft_consensus.cc:2804] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.964598 19885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:46.965221 19885 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.964851 20080 leader_election.cc:290] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:46.970453 19960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:46.971100 19960 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.971529 19958 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:46.972729 20000 leader_election.cc:304] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:46.965689 20075 raft_consensus.cc:385] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.966181 19850 leader_election.cc:304] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.975438 20075 raft_consensus.cc:740] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.976612 19882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:46.976912 19885 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:46.960846 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:46.964080 20101 raft_consensus.cc:697] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:46.978027 19884 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.978529 19998 leader_election.cc:304] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:46.979065 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:46.979789 20111 raft_consensus.cc:2804] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.980346 20111 raft_consensus.cc:697] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:46.965610 20033 raft_consensus.cc:2393] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79b099cf2a294ddea68376a11141032f in current term 1: Already voted for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in this term.
I20260709 13:04:46.977183 20075 consensus_queue.cc:260] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.981179 20111 consensus_queue.cc:237] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:46.981858 20097 raft_consensus.cc:2804] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.982254 20097 raft_consensus.cc:493] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.982636 20097 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.983553 19960 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:46.978536 20101 consensus_queue.cc:237] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.985051 20113 raft_consensus.cc:2804] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.985231 19924 leader_election.cc:304] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79b099cf2a294ddea68376a11141032f; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:46.987084 20083 raft_consensus.cc:2749] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.969270 20076 tablet_bootstrap.cc:654] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:46.992138 20075 ts_tablet_manager.cc:1434] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 13:04:46.993480 20097 raft_consensus.cc:515] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.993927 19959 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:46.995930 20075 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:46.997849 20074 raft_consensus.cc:359] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:46.998605 20074 raft_consensus.cc:385] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.998903 20074 raft_consensus.cc:740] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.999881 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:47.000746 19958 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.000968 19883 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.002002 19884 raft_consensus.cc:2468] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.006978 20113 raft_consensus.cc:697] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.005194 20097 leader_election.cc:290] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.013933 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.014606 20033 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.016197 20113 consensus_queue.cc:237] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.017189 20075 tablet_bootstrap.cc:654] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.022070 19782 catalog_manager.cc:5697] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.012960 20074 consensus_queue.cc:260] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.026338 19958 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:47.027022 20074 ts_tablet_manager.cc:1434] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:04:47.027772 19851 leader_election.cc:304] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.028069 20074 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:47.029506 20097 raft_consensus.cc:2804] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.030025 20097 raft_consensus.cc:697] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:47.030745 20097 consensus_queue.cc:237] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.024163 20033 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:47.036733 20074 tablet_bootstrap.cc:654] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.035722 19783 catalog_manager.cc:5697] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.037911 20076 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:47.039150 20076 ts_tablet_manager.cc:1403] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.005s
I20260709 13:04:47.043210 20076 raft_consensus.cc:359] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.044843 20076 raft_consensus.cc:385] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.045300 20076 raft_consensus.cc:740] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.047564 20075 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:47.048312 20075 ts_tablet_manager.cc:1403] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.006s
I20260709 13:04:47.049700 20076 consensus_queue.cc:260] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.051535 20076 ts_tablet_manager.cc:1434] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:04:47.051311 20075 raft_consensus.cc:359] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.052146 20075 raft_consensus.cc:385] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.052562 20076 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:47.052485 20075 raft_consensus.cc:740] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.053548 20075 consensus_queue.cc:260] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.055701 20097 raft_consensus.cc:493] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.056270 20097 raft_consensus.cc:515] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.058573 20097 leader_election.cc:290] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.059415 20076 tablet_bootstrap.cc:654] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.060145 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:47.061246 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
W20260709 13:04:47.062669 19850 leader_election.cc:343] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:47.063422 19851 leader_election.cc:343] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:47.063444 20075 ts_tablet_manager.cc:1434] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:04:47.063933 19851 leader_election.cc:304] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa705cbdb57641be99f49d5e3c4225e6; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f
I20260709 13:04:47.065129 20075 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:47.065178 20097 raft_consensus.cc:2749] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:47.064783 19781 catalog_manager.cc:5697] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.068367 19782 catalog_manager.cc:5697] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.070436 19782 catalog_manager.cc:5697] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.073864 20075 tablet_bootstrap.cc:654] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.078713 20074 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:47.079173 20074 ts_tablet_manager.cc:1403] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.004s
I20260709 13:04:47.082697 20076 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:47.083217 20076 ts_tablet_manager.cc:1403] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.008s
I20260709 13:04:47.085785 20076 raft_consensus.cc:359] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.086444 20076 raft_consensus.cc:385] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.086714 20076 raft_consensus.cc:740] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.087782 20074 raft_consensus.cc:359] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.088516 20074 raft_consensus.cc:385] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.088796 20074 raft_consensus.cc:740] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.089553 20074 consensus_queue.cc:260] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.089598 20076 consensus_queue.cc:260] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.091473 20075 tablet_bootstrap.cc:492] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:47.091584 20076 ts_tablet_manager.cc:1434] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:04:47.092633 20076 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:47.091589 20074 ts_tablet_manager.cc:1434] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:04:47.094717 20075 ts_tablet_manager.cc:1403] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.005s
I20260709 13:04:47.095058 20074 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:47.097522 20075 raft_consensus.cc:359] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.098269 20075 raft_consensus.cc:385] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.098596 20075 raft_consensus.cc:740] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.099104 20076 tablet_bootstrap.cc:654] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.099445 20075 consensus_queue.cc:260] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.101120 20074 tablet_bootstrap.cc:654] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:47.102330 20075 ts_tablet_manager.cc:1434] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:47.110525 20076 tablet_bootstrap.cc:492] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:47.110723 20074 tablet_bootstrap.cc:492] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:47.111121 20076 ts_tablet_manager.cc:1403] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:04:47.111269 20074 ts_tablet_manager.cc:1403] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:04:47.113240 20076 raft_consensus.cc:359] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.113950 20076 raft_consensus.cc:385] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.114390 20076 raft_consensus.cc:740] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.113977 20074 raft_consensus.cc:359] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.114871 20074 raft_consensus.cc:385] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:47.115161 20074 raft_consensus.cc:740] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:47.115268 20076 consensus_queue.cc:260] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.115892 20074 consensus_queue.cc:260] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.117401 20076 ts_tablet_manager.cc:1434] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:04:47.118216 20074 ts_tablet_manager.cc:1434] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:47.135615 20083 raft_consensus.cc:493] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.136094 20083 raft_consensus.cc:515] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.137681 20083 leader_election.cc:290] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.138554 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b0cf2c1fe954feba34a9cb23532cbca" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:47.138664 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b0cf2c1fe954feba34a9cb23532cbca" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:47.139187 19884 raft_consensus.cc:2468] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:47.139283 20033 raft_consensus.cc:2468] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:47.140187 19924 leader_election.cc:304] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.140888 20083 raft_consensus.cc:2804] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.141170 20083 raft_consensus.cc:493] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.141477 20083 raft_consensus.cc:3060] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.146345 20083 raft_consensus.cc:515] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.147794 20083 leader_election.cc:290] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.148595 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b0cf2c1fe954feba34a9cb23532cbca" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.148927 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b0cf2c1fe954feba34a9cb23532cbca" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.149278 19884 raft_consensus.cc:3060] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.149549 20033 raft_consensus.cc:3060] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.156324 19884 raft_consensus.cc:2468] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:47.156333 20033 raft_consensus.cc:2468] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:47.157840 19924 leader_election.cc:304] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.158716 20083 raft_consensus.cc:2804] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.159590 20083 raft_consensus.cc:697] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:47.160264 20083 consensus_queue.cc:237] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.167214 19782 catalog_manager.cc:5697] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 1 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.258893 20113 raft_consensus.cc:493] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.259663 20113 raft_consensus.cc:515] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.261567 20113 leader_election.cc:290] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:47.262516 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153d1a15c28f47b8b822a41ab43d1adc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:47.262748 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153d1a15c28f47b8b822a41ab43d1adc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:47.263293 19884 raft_consensus.cc:2468] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:47.263366 19958 raft_consensus.cc:2468] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:47.264482 19998 leader_election.cc:304] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.265218 20113 raft_consensus.cc:2804] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.265560 20113 raft_consensus.cc:493] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.265836 20113 raft_consensus.cc:3060] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.270849 20113 raft_consensus.cc:515] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.272433 20113 leader_election.cc:290] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:47.273178 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153d1a15c28f47b8b822a41ab43d1adc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.273455 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153d1a15c28f47b8b822a41ab43d1adc" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:47.273878 19884 raft_consensus.cc:3060] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.273993 19958 raft_consensus.cc:3060] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.280889 19884 raft_consensus.cc:2468] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.280889 19958 raft_consensus.cc:2468] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.282155 20000 leader_election.cc:304] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:47.282795 20113 raft_consensus.cc:2804] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.283344 20113 raft_consensus.cc:697] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.284087 20113 consensus_queue.cc:237] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.291723 19782 catalog_manager.cc:5697] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.297333 20097 consensus_queue.cc:1048] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:47.309975 20104 consensus_queue.cc:1048] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.320624 20104 consensus_queue.cc:1048] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.333807 20097 consensus_queue.cc:1048] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.365414 20113 raft_consensus.cc:493] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.365959 20113 raft_consensus.cc:515] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.367722 20113 leader_election.cc:290] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:47.368505 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:47.368711 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:47.369158 19884 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:47.369415 19958 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:47.370184 19998 leader_election.cc:304] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.370924 20113 raft_consensus.cc:2804] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:47.371301 20113 raft_consensus.cc:493] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.371582 20113 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.376479 20113 raft_consensus.cc:515] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.378207 20113 leader_election.cc:290] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:47.378932 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.379148 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:47.379484 19884 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.379755 19958 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:47.384649 19884 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.384721 19958 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:47.385888 19998 leader_election.cc:304] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.386644 20113 raft_consensus.cc:2804] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:47.387050 20113 raft_consensus.cc:697] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.387830 20113 consensus_queue.cc:237] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.390846 20101 consensus_queue.cc:1048] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.396116 19782 catalog_manager.cc:5697] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:47.405498 20113 consensus_queue.cc:1048] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.435804 20118 consensus_queue.cc:1048] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.448132 20113 consensus_queue.cc:1048] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.469100 20033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d08efba25a74cb69708233eef807713"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:47.469748 20033 raft_consensus.cc:606] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:47.474854 20033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:47.475368 20033 raft_consensus.cc:606] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:47.485824 20033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:47.486374 20033 raft_consensus.cc:606] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:47.499436 19884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:55528
I20260709 13:04:47.499924 19884 raft_consensus.cc:606] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:47.505714 20113 consensus_queue.cc:1048] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.509095 20033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "713e842bd00941b4b931176299bc5ada"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:47.509703 20033 raft_consensus.cc:606] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:47.512885 20118 raft_consensus.cc:993] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.513338 20118 raft_consensus.cc:1081] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.516921 19959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:04:47.517450 19959 raft_consensus.cc:493] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:47.517834 19959 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.519131 20118 consensus_queue.cc:1048] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.524389 19959 raft_consensus.cc:515] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.528966 19959 leader_election.cc:290] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.530102 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.529124 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c8e89b7ec540cb9eda24fd98ef7acc" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.530777 20033 raft_consensus.cc:3055] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:47.531119 19884 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.531165 20033 raft_consensus.cc:740] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.532109 20033 consensus_queue.cc:260] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.533208 20033 raft_consensus.cc:3060] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.537156 19884 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.538447 19924 leader_election.cc:304] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.539199 20083 raft_consensus.cc:2804] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:47.539336 20033 raft_consensus.cc:2468] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.539719 20083 raft_consensus.cc:697] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:47.540591 20083 consensus_queue.cc:237] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.546841 19781 catalog_manager.cc:5697] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.605453 20097 consensus_queue.cc:1048] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.617178 20104 consensus_queue.cc:1048] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.624660 20096 raft_consensus.cc:993] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.625239 20104 raft_consensus.cc:1081] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.627045 19959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39696
I20260709 13:04:47.627562 19959 raft_consensus.cc:493] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:47.627859 19959 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.633241 19959 raft_consensus.cc:515] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.635321 19959 leader_election.cc:290] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.636210 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.636494 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfbbbe561e014873b8e6fe086f3a8b31" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.636983 19884 raft_consensus.cc:3055] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:47.637171 20033 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.637427 19884 raft_consensus.cc:740] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:47.638350 19884 consensus_queue.cc:260] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.639725 19884 raft_consensus.cc:3060] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.643266 20033 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.644361 19925 leader_election.cc:304] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:47.645135 20083 raft_consensus.cc:2804] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:47.645248 19884 raft_consensus.cc:2468] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.645630 20083 raft_consensus.cc:697] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:47.646584 20083 consensus_queue.cc:237] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.653364 19783 catalog_manager.cc:5697] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.672435 20083 consensus_queue.cc:1048] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.680403 20131 consensus_queue.cc:1048] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.694965 20118 consensus_queue.cc:1048] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.708422 20111 consensus_queue.cc:1048] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.902235 20113 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.905438 20113 raft_consensus.cc:993] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:47.905894 20101 raft_consensus.cc:1081] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:47.907527 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:55512
I20260709 13:04:47.908162 19884 raft_consensus.cc:493] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:47.908553 19884 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.915599 19884 raft_consensus.cc:515] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.917608 20118 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.918727 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:47.919351 19959 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.920153 19884 leader_election.cc:290] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.922456 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d601750e621f4e9a83d1153e5b946a7e" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.923084 20033 raft_consensus.cc:3055] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:47.923393 20033 raft_consensus.cc:740] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.924185 20033 consensus_queue.cc:260] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.925148 20118 raft_consensus.cc:993] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.925503 20033 raft_consensus.cc:3060] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.925699 20118 raft_consensus.cc:1081] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:47.928612 19958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "713e842bd00941b4b931176299bc5ada"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:04:47.929200 19958 raft_consensus.cc:493] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:47.929606 19958 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.929979 19959 raft_consensus.cc:2468] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:47.931360 19851 leader_election.cc:304] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.932268 20097 raft_consensus.cc:2804] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:47.932798 20097 raft_consensus.cc:697] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:47.933630 20097 consensus_queue.cc:237] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.935542 20033 raft_consensus.cc:2468] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:47.936550 19958 raft_consensus.cc:515] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.938784 19958 leader_election.cc:290] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:47.939594 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:47.940249 19884 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.941632 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:47.941637 19781 catalog_manager.cc:5697] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.942433 20033 raft_consensus.cc:3055] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:47.944352 20033 raft_consensus.cc:740] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:47.945235 20033 consensus_queue.cc:260] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.946635 20033 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:47.947747 19884 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.949052 19924 leader_election.cc:304] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:47.949885 20154 raft_consensus.cc:2804] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:47.950356 20154 raft_consensus.cc:697] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:47.951174 20154 consensus_queue.cc:237] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:47.953768 20033 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:47.957528 19781 catalog_manager.cc:5697] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:48.002907 20118 raft_consensus.cc:993] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:48.003458 20113 raft_consensus.cc:1081] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:48.004866 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d08efba25a74cb69708233eef807713"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:55512
I20260709 13:04:48.005416 19884 raft_consensus.cc:493] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:48.005724 19884 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:48.011190 19884 raft_consensus.cc:515] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:48.012817 19884 leader_election.cc:290] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:48.013756 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:48.013922 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d08efba25a74cb69708233eef807713" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:48.014492 19958 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:48.014531 20033 raft_consensus.cc:3055] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:48.015025 20033 raft_consensus.cc:740] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:48.015866 20033 consensus_queue.cc:260] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:48.017117 20033 raft_consensus.cc:3060] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:48.022008 19958 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:48.023443 19851 leader_election.cc:304] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:48.024333 20097 raft_consensus.cc:2804] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:48.024453 20033 raft_consensus.cc:2468] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:48.024619 20034 raft_consensus.cc:1275] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.024814 20097 raft_consensus.cc:697] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:48.025825 20097 consensus_queue.cc:237] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } }
I20260709 13:04:48.026239 20154 consensus_queue.cc:1048] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.033532 19781 catalog_manager.cc:5697] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:48.038064 19884 raft_consensus.cc:1275] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.040009 20131 consensus_queue.cc:1048] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.221225 20033 raft_consensus.cc:1275] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.222987 20131 consensus_queue.cc:1048] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:48.234369 19884 raft_consensus.cc:1275] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.236685 20083 consensus_queue.cc:1048] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:48.352352 19958 raft_consensus.cc:1275] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.354031 20097 consensus_queue.cc:1048] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.366288 20033 raft_consensus.cc:1275] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.367527 20097 consensus_queue.cc:1048] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.405548 19884 raft_consensus.cc:1275] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.407002 20083 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.418457 20033 raft_consensus.cc:1275] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.419924 20083 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.511502 19958 raft_consensus.cc:1275] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.512717 20097 consensus_queue.cc:1048] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:48.523983 20033 raft_consensus.cc:1275] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:48.526782 20163 consensus_queue.cc:1048] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:50.551839 19783 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36808:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:50.554801 19783 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:50.614375 19873 tablet_service.cc:1511] Processing CreateTablet for tablet 8849f14f2ac14be59c3f5ce389ee5504 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:50.615880 19946 tablet_service.cc:1511] Processing CreateTablet for tablet 8849f14f2ac14be59c3f5ce389ee5504 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:50.616390 19873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849f14f2ac14be59c3f5ce389ee5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.617705 19946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849f14f2ac14be59c3f5ce389ee5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.618615 19871 tablet_service.cc:1511] Processing CreateTablet for tablet 541c795c5782411c8c8165273ea72e97 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:50.620150 19871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541c795c5782411c8c8165273ea72e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.622691 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 8849f14f2ac14be59c3f5ce389ee5504 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:50.624348 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849f14f2ac14be59c3f5ce389ee5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.631392 19950 tablet_service.cc:1511] Processing CreateTablet for tablet 541c795c5782411c8c8165273ea72e97 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:50.633160 19950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541c795c5782411c8c8165273ea72e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.639590 20024 tablet_service.cc:1511] Processing CreateTablet for tablet 541c795c5782411c8c8165273ea72e97 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:50.641165 20024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 541c795c5782411c8c8165273ea72e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.641827 19872 tablet_service.cc:1511] Processing CreateTablet for tablet 0d98942d5e644d14965245d02d872469 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:50.643268 19872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d98942d5e644d14965245d02d872469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.644258 19949 tablet_service.cc:1511] Processing CreateTablet for tablet 0d98942d5e644d14965245d02d872469 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:50.645715 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d98942d5e644d14965245d02d872469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.650849 19875 tablet_service.cc:1511] Processing CreateTablet for tablet e5ba5601b329454a92b5fc4e4276714c (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:50.651995 20019 tablet_service.cc:1511] Processing CreateTablet for tablet 0d98942d5e644d14965245d02d872469 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:50.653606 19875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba5601b329454a92b5fc4e4276714c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.655679 19948 tablet_service.cc:1511] Processing CreateTablet for tablet e5ba5601b329454a92b5fc4e4276714c (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:50.656761 20020 tablet_service.cc:1511] Processing CreateTablet for tablet e5ba5601b329454a92b5fc4e4276714c (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:50.657406 19948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba5601b329454a92b5fc4e4276714c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.661648 20019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d98942d5e644d14965245d02d872469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.662573 19874 tablet_service.cc:1511] Processing CreateTablet for tablet 357cf49a168243adb1cd4a9935593aa9 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:50.663554 20020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ba5601b329454a92b5fc4e4276714c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.663959 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357cf49a168243adb1cd4a9935593aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.684523 19947 tablet_service.cc:1511] Processing CreateTablet for tablet 357cf49a168243adb1cd4a9935593aa9 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:50.686474 19947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357cf49a168243adb1cd4a9935593aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.713358 20177 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:50.720045 20178 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:50.732601 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 357cf49a168243adb1cd4a9935593aa9 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:50.738487 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357cf49a168243adb1cd4a9935593aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.738294 20023 tablet_service.cc:1511] Processing CreateTablet for tablet a05ac482d0e14fc99e404ff234bd53fa (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:50.734694 19870 tablet_service.cc:1511] Processing CreateTablet for tablet a05ac482d0e14fc99e404ff234bd53fa (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:50.739204 19946 tablet_service.cc:1511] Processing CreateTablet for tablet a05ac482d0e14fc99e404ff234bd53fa (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:50.739742 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05ac482d0e14fc99e404ff234bd53fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.740370 19870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05ac482d0e14fc99e404ff234bd53fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.740477 19946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05ac482d0e14fc99e404ff234bd53fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.757323 20177 tablet_bootstrap.cc:654] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.766937 20178 tablet_bootstrap.cc:654] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.770840 19874 tablet_service.cc:1511] Processing CreateTablet for tablet 3b7f2c14823c4d7c9350fe2546e4141d (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:50.772739 19874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7f2c14823c4d7c9350fe2546e4141d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.785035 19946 tablet_service.cc:1511] Processing CreateTablet for tablet 3b7f2c14823c4d7c9350fe2546e4141d (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:50.786896 19946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7f2c14823c4d7c9350fe2546e4141d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.795883 20178 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:50.796491 20178 ts_tablet_manager.cc:1403] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.077s	user 0.021s	sys 0.009s
I20260709 13:04:50.799204 20178 raft_consensus.cc:359] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.799770 20178 raft_consensus.cc:385] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.800028 20178 raft_consensus.cc:740] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.800770 20178 consensus_queue.cc:260] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.803486 20178 ts_tablet_manager.cc:1434] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.007s
I20260709 13:04:50.804512 20178 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:50.808612 20022 tablet_service.cc:1511] Processing CreateTablet for tablet 3b7f2c14823c4d7c9350fe2546e4141d (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:50.810355 20022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b7f2c14823c4d7c9350fe2546e4141d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.811265 20178 tablet_bootstrap.cc:654] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.817662 19870 tablet_service.cc:1511] Processing CreateTablet for tablet 3a9b4b7f33f94c29b99a114301ee76d5 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:50.819190 19870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a9b4b7f33f94c29b99a114301ee76d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.819550 19948 tablet_service.cc:1511] Processing CreateTablet for tablet 3a9b4b7f33f94c29b99a114301ee76d5 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:50.820801 19948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a9b4b7f33f94c29b99a114301ee76d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.841866 20021 tablet_service.cc:1511] Processing CreateTablet for tablet 3a9b4b7f33f94c29b99a114301ee76d5 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:50.843505 20021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a9b4b7f33f94c29b99a114301ee76d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.852816 19872 tablet_service.cc:1511] Processing CreateTablet for tablet cbafbdd4378a47cd8fdbb28076f91c85 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 13:04:50.852761 19831 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:50.854421 19872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbafbdd4378a47cd8fdbb28076f91c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.871956 20179 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:50.873104 19949 tablet_service.cc:1511] Processing CreateTablet for tablet cbafbdd4378a47cd8fdbb28076f91c85 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:50.874265 20177 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:50.874756 20177 ts_tablet_manager.cc:1403] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.162s	user 0.018s	sys 0.037s
I20260709 13:04:50.875082 19949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbafbdd4378a47cd8fdbb28076f91c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.877867 20177 raft_consensus.cc:359] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.878707 20177 raft_consensus.cc:385] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.879052 20177 raft_consensus.cc:740] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.879976 20177 consensus_queue.cc:260] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.880695 20023 tablet_service.cc:1511] Processing CreateTablet for tablet cbafbdd4378a47cd8fdbb28076f91c85 (DEFAULT_TABLE table=soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:50.874044 20178 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:50.881976 20023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbafbdd4378a47cd8fdbb28076f91c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:50.882296 20178 ts_tablet_manager.cc:1403] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.078s	user 0.007s	sys 0.009s
I20260709 13:04:50.889858 20178 raft_consensus.cc:359] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.890338 20177 ts_tablet_manager.cc:1434] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:04:50.897845 20178 raft_consensus.cc:385] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.896154 20179 tablet_bootstrap.cc:654] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.898200 20178 raft_consensus.cc:740] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.899214 20177 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:50.901885 20178 consensus_queue.cc:260] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.904481 20178 ts_tablet_manager.cc:1434] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 13:04:50.905697 20178 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:50.908031 20179 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:50.908730 20179 ts_tablet_manager.cc:1403] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.003s
I20260709 13:04:50.909504 20177 tablet_bootstrap.cc:654] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.914325 20179 raft_consensus.cc:359] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.915112 20179 raft_consensus.cc:385] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.915429 20179 raft_consensus.cc:740] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.915438 20178 tablet_bootstrap.cc:654] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.916414 20179 consensus_queue.cc:260] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.917021 20177 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:50.919217 20179 ts_tablet_manager.cc:1434] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:50.917625 20177 ts_tablet_manager.cc:1403] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.007s
I20260709 13:04:50.920286 20179 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:50.923413 20177 raft_consensus.cc:359] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.925662 20182 raft_consensus.cc:493] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.926203 20182 raft_consensus.cc:515] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.927832 20179 tablet_bootstrap.cc:654] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.928745 20182 leader_election.cc:290] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:50.930892 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:50.924254 20177 raft_consensus.cc:385] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.932693 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:50.935911 19884 raft_consensus.cc:2468] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:50.937779 19924 leader_election.cc:304] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
W20260709 13:04:50.938506 19925 leader_election.cc:343] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.938639 20178 tablet_bootstrap.cc:492] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:50.938715 20182 raft_consensus.cc:2804] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.939268 20182 raft_consensus.cc:493] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.939392 20178 ts_tablet_manager.cc:1403] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.003s
I20260709 13:04:50.939642 20182 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.942727 20161 raft_consensus.cc:493] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.943481 20161 raft_consensus.cc:515] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.936085 20177 raft_consensus.cc:740] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.945807 20161 leader_election.cc:290] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:50.945945 20179 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:50.942616 20178 raft_consensus.cc:359] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.946440 20178 raft_consensus.cc:385] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.946491 20179 ts_tablet_manager.cc:1403] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.011s
I20260709 13:04:50.946843 20178 raft_consensus.cc:740] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.947039 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:50.947659 20178 consensus_queue.cc:260] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
W20260709 13:04:50.948812 19851 leader_election.cc:343] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.948541 20182 raft_consensus.cc:515] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.949723 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:50.950176 20178 ts_tablet_manager.cc:1434] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:50.950004 20179 raft_consensus.cc:359] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.950901 20179 raft_consensus.cc:385] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:50.950991 19850 leader_election.cc:343] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.951229 20179 raft_consensus.cc:740] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.951609 19850 leader_election.cc:304] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa705cbdb57641be99f49d5e3c4225e6; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f
I20260709 13:04:50.952335 20161 raft_consensus.cc:2749] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.952000 20179 consensus_queue.cc:260] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.953583 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:50.954185 19884 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.954517 20179 ts_tablet_manager.cc:1434] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:50.955204 20178 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:50.955579 20179 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:50.946462 20177 consensus_queue.cc:260] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.962586 20179 tablet_bootstrap.cc:654] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.968083 20177 ts_tablet_manager.cc:1434] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.002s
I20260709 13:04:50.969276 20177 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:50.971544 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
W20260709 13:04:50.975253 19925 leader_election.cc:343] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.966466 20178 tablet_bootstrap.cc:654] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.980746 20182 leader_election.cc:290] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:50.973862 19884 raft_consensus.cc:2468] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:50.982371 19924 leader_election.cc:304] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb
I20260709 13:04:50.983700 20182 raft_consensus.cc:2804] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.984570 20182 raft_consensus.cc:697] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:50.985512 20182 consensus_queue.cc:237] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.987521 20179 tablet_bootstrap.cc:492] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:50.992321 20179 ts_tablet_manager.cc:1403] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.012s
I20260709 13:04:50.991117 20177 tablet_bootstrap.cc:654] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.995551 20179 raft_consensus.cc:359] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:50.995975 20178 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:50.996330 20179 raft_consensus.cc:385] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.996688 20178 ts_tablet_manager.cc:1403] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.005s
I20260709 13:04:50.996711 20179 raft_consensus.cc:740] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.997870 20179 consensus_queue.cc:260] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.000296 20179 ts_tablet_manager.cc:1434] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:04:51.001368 20179 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.003757 20178 raft_consensus.cc:359] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.008002 20179 tablet_bootstrap.cc:654] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.004572 20178 raft_consensus.cc:385] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.009034 20178 raft_consensus.cc:740] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.009879 20178 consensus_queue.cc:260] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.011592 20113 raft_consensus.cc:493] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.012089 20113 raft_consensus.cc:515] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.012701 20178 ts_tablet_manager.cc:1434] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:04:51.014320 20113 leader_election.cc:290] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.014333 20178 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:51.016301 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.018011 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.018718 19884 raft_consensus.cc:2468] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
W20260709 13:04:51.020079 20000 leader_election.cc:343] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.020444 20179 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.021152 20179 ts_tablet_manager.cc:1403] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:04:51.021548 19998 leader_election.cc:304] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.022836 20113 raft_consensus.cc:493] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.023555 20178 tablet_bootstrap.cc:654] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.023283 20113 raft_consensus.cc:515] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.024495 20179 raft_consensus.cc:359] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.025322 20179 raft_consensus.cc:385] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.025679 20179 raft_consensus.cc:740] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.026573 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.026535 20179 consensus_queue.cc:260] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.033126 20113 leader_election.cc:290] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.033674 20113 raft_consensus.cc:2804] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.033988 20113 raft_consensus.cc:493] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.034308 20113 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.034478 19782 catalog_manager.cc:5697] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 1 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.039139 20177 tablet_bootstrap.cc:492] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.039757 20177 ts_tablet_manager.cc:1403] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.002s
I20260709 13:04:51.051160 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.051923 19884 raft_consensus.cc:2393] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bc1ec1548ac4dbc933b527c9d4103fb in current term 1: Already voted for candidate 79b099cf2a294ddea68376a11141032f in this term.
I20260709 13:04:51.052979 20178 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:51.053231 19998 leader_election.cc:304] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb; no voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:51.053820 20178 ts_tablet_manager.cc:1403] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.005s
I20260709 13:04:51.048972 20177 raft_consensus.cc:359] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.055598 20177 raft_consensus.cc:385] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.055972 20177 raft_consensus.cc:740] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.056442 20179 ts_tablet_manager.cc:1434] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.018s
I20260709 13:04:51.056898 20177 consensus_queue.cc:260] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.054569 20194 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.057760 20179 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.059619 20177 ts_tablet_manager.cc:1434] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.019s	user 0.001s	sys 0.005s
I20260709 13:04:51.060776 20177 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.065025 20179 tablet_bootstrap.cc:654] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.065739 20194 raft_consensus.cc:2749] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.057796 20178 raft_consensus.cc:359] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.067766 20178 raft_consensus.cc:385] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.067938 20177 tablet_bootstrap.cc:654] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.068209 20178 raft_consensus.cc:740] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.069267 20178 consensus_queue.cc:260] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.050591 20113 raft_consensus.cc:515] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.073475 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.074038 19884 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.074669 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.075754 20178 ts_tablet_manager.cc:1434] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 13:04:51.076073 20000 leader_election.cc:343] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.076817 20178 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:51.077018 20179 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.077517 20179 ts_tablet_manager.cc:1403] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:04:51.080914 19884 raft_consensus.cc:2468] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:51.080713 20179 raft_consensus.cc:359] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.081782 20179 raft_consensus.cc:385] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.082151 20179 raft_consensus.cc:740] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.082195 19998 leader_election.cc:304] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.083117 20194 raft_consensus.cc:2804] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.073482 20113 leader_election.cc:290] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.083609 20194 raft_consensus.cc:697] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:51.083139 20179 consensus_queue.cc:260] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.084964 20194 consensus_queue.cc:237] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.086434 20179 ts_tablet_manager.cc:1434] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:04:51.087500 20179 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.091315 20177 tablet_bootstrap.cc:492] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.091851 20177 ts_tablet_manager.cc:1403] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.006s
I20260709 13:04:51.093513 20178 tablet_bootstrap.cc:654] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.094825 20179 tablet_bootstrap.cc:654] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.095148 20177 raft_consensus.cc:359] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.095957 20177 raft_consensus.cc:385] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.096267 20177 raft_consensus.cc:740] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.097036 20177 consensus_queue.cc:260] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.099550 20177 ts_tablet_manager.cc:1434] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:51.100582 20177 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.107689 20177 tablet_bootstrap.cc:654] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.125471 19783 catalog_manager.cc:5697] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.143846 20178 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:51.144675 20178 ts_tablet_manager.cc:1403] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.027s
I20260709 13:04:51.149224 20178 raft_consensus.cc:359] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.150019 20178 raft_consensus.cc:385] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.153782 20178 raft_consensus.cc:740] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.154982 20178 consensus_queue.cc:260] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.157162 20179 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.157792 20179 ts_tablet_manager.cc:1403] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.071s	user 0.025s	sys 0.035s
I20260709 13:04:51.158900 20177 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.159493 20177 ts_tablet_manager.cc:1403] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.029s
I20260709 13:04:51.160566 20178 ts_tablet_manager.cc:1434] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:04:51.161190 20179 raft_consensus.cc:359] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.161773 20178 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:51.162127 20179 raft_consensus.cc:385] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.162496 20179 raft_consensus.cc:740] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.162618 20177 raft_consensus.cc:359] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.163324 20177 raft_consensus.cc:385] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.163309 20179 consensus_queue.cc:260] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.163885 20177 raft_consensus.cc:740] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.164995 20177 consensus_queue.cc:260] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.165954 20179 ts_tablet_manager.cc:1434] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:51.167189 20179 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.172930 20177 ts_tablet_manager.cc:1434] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:51.173043 20178 tablet_bootstrap.cc:654] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.173679 20179 tablet_bootstrap.cc:654] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.174225 20177 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.177398 20161 raft_consensus.cc:493] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.177475 20182 raft_consensus.cc:493] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.178020 20161 raft_consensus.cc:515] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.178298 20182 raft_consensus.cc:515] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.180222 20161 leader_election.cc:290] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.182121 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d98942d5e644d14965245d02d872469" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.183014 19958 raft_consensus.cc:2468] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:51.186118 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.187027 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d98942d5e644d14965245d02d872469" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.187714 20034 raft_consensus.cc:2468] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:51.187925 20182 leader_election.cc:290] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.189649 19851 leader_election.cc:304] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.190570 20161 raft_consensus.cc:2804] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.190922 20161 raft_consensus.cc:493] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.191252 20161 raft_consensus.cc:3060] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.191792 20179 tablet_bootstrap.cc:492] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.192322 20179 ts_tablet_manager.cc:1403] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 13:04:51.185185 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.194013 19884 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:51.194986 20177 tablet_bootstrap.cc:654] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.187083 20033 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:51.195467 20179 raft_consensus.cc:359] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.196231 20179 raft_consensus.cc:385] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.196499 20179 raft_consensus.cc:740] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.197209 19925 leader_election.cc:304] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:51.197208 20179 consensus_queue.cc:260] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.199097 20083 raft_consensus.cc:2804] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.201373 20083 raft_consensus.cc:493] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.201719 20083 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.203178 20179 ts_tablet_manager.cc:1434] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.007s
I20260709 13:04:51.204242 20179 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.197875 20178 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:51.207659 20178 ts_tablet_manager.cc:1403] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.046s	user 0.020s	sys 0.004s
I20260709 13:04:51.209594 20188 raft_consensus.cc:493] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.210150 20188 raft_consensus.cc:515] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.210633 20179 tablet_bootstrap.cc:654] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.209870 20083 raft_consensus.cc:515] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.211388 20178 raft_consensus.cc:359] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.212183 20178 raft_consensus.cc:385] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.212466 20178 raft_consensus.cc:740] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.212762 20188 leader_election.cc:290] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.216490 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.216550 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.217506 20033 raft_consensus.cc:2468] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:51.213126 20178 consensus_queue.cc:260] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.219311 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
W20260709 13:04:51.219827 19851 leader_election.cc:343] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:51.220124 19884 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.213176 20083 leader_election.cc:290] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.231864 19850 leader_election.cc:304] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.234975 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.235622 20033 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.236349 20188 raft_consensus.cc:2804] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.236773 20188 raft_consensus.cc:493] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.236955 20161 raft_consensus.cc:515] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.244192 20161 leader_election.cc:290] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.247740 19884 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:51.248400 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d98942d5e644d14965245d02d872469" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.249011 19958 raft_consensus.cc:3060] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.237102 20188 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.251546 19924 leader_election.cc:304] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.253371 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d98942d5e644d14965245d02d872469" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.253999 20182 raft_consensus.cc:2804] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.254559 20182 raft_consensus.cc:697] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:51.246075 20083 raft_consensus.cc:493] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.255780 20182 consensus_queue.cc:237] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.254029 20032 raft_consensus.cc:3060] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.258170 20033 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:51.255985 20083 raft_consensus.cc:515] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.259475 20177 tablet_bootstrap.cc:492] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.260041 20177 ts_tablet_manager.cc:1403] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.086s	user 0.021s	sys 0.009s
I20260709 13:04:51.261957 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.262724 19884 raft_consensus.cc:2393] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79b099cf2a294ddea68376a11141032f in current term 1: Already voted for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in this term.
I20260709 13:04:51.262815 20179 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.263309 20179 ts_tablet_manager.cc:1403] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.014s
I20260709 13:04:51.263221 20177 raft_consensus.cc:359] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.264075 20177 raft_consensus.cc:385] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.264364 20177 raft_consensus.cc:740] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.265110 20032 raft_consensus.cc:2468] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.265645 20178 ts_tablet_manager.cc:1434] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:04:51.265080 20177 consensus_queue.cc:260] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.270700 19850 leader_election.cc:304] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.266296 20083 leader_election.cc:290] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.271414 20178 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:51.267704 20188 raft_consensus.cc:515] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.271510 20179 raft_consensus.cc:359] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.271745 20209 raft_consensus.cc:2804] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.279358 20209 raft_consensus.cc:697] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:51.278091 20179 raft_consensus.cc:385] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.281599 20179 raft_consensus.cc:740] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.286669 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.287312 19959 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.290271 20179 consensus_queue.cc:260] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.277232 20083 raft_consensus.cc:493] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.282274 20188 leader_election.cc:290] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.290548 20209 consensus_queue.cc:237] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.291612 19958 raft_consensus.cc:2468] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.293267 20179 ts_tablet_manager.cc:1434] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.005s
I20260709 13:04:51.277634 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.295550 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.297407 20083 raft_consensus.cc:515] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.302521 20033 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.303264 20179 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: Bootstrap starting.
I20260709 13:04:51.309815 19925 leader_election.cc:304] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79b099cf2a294ddea68376a11141032f; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:51.317723 20033 raft_consensus.cc:2468] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.319954 20211 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.320695 19850 leader_election.cc:304] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.324469 19781 catalog_manager.cc:5697] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 1 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.325954 20083 leader_election.cc:290] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.326969 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05ac482d0e14fc99e404ff234bd53fa" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.327345 19959 raft_consensus.cc:2468] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.327733 19884 raft_consensus.cc:2468] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:51.328303 20209 raft_consensus.cc:2804] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.328982 20209 raft_consensus.cc:697] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:51.329468 20177 ts_tablet_manager.cc:1434] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.005s
I20260709 13:04:51.329703 19924 leader_election.cc:304] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.329363 19782 catalog_manager.cc:5697] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.329850 20209 consensus_queue.cc:237] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.330631 20177 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.333153 20179 tablet_bootstrap.cc:654] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.334681 20211 raft_consensus.cc:2749] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:51.336054 20182 raft_consensus.cc:2804] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.336491 20182 raft_consensus.cc:493] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.336886 20182 raft_consensus.cc:3060] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.337325 20178 tablet_bootstrap.cc:654] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.337775 20177 tablet_bootstrap.cc:654] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.342607 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05ac482d0e14fc99e404ff234bd53fa" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.343256 20033 raft_consensus.cc:2468] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 0.
I20260709 13:04:51.346405 20179 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: No bootstrap required, opened a new log
I20260709 13:04:51.348699 20179 ts_tablet_manager.cc:1403] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 13:04:51.354944 20178 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:51.354990 20182 raft_consensus.cc:515] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.356051 20178 ts_tablet_manager.cc:1403] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.006s
I20260709 13:04:51.358059 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05ac482d0e14fc99e404ff234bd53fa" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.358675 19884 raft_consensus.cc:3060] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.359153 20178 raft_consensus.cc:359] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.359894 20178 raft_consensus.cc:385] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.360205 20178 raft_consensus.cc:740] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.360906 20178 consensus_queue.cc:260] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.363267 20178 ts_tablet_manager.cc:1434] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:51.364248 20178 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: Bootstrap starting.
I20260709 13:04:51.364468 20179 raft_consensus.cc:359] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.364741 19782 catalog_manager.cc:5697] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.365516 20179 raft_consensus.cc:385] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.366988 20179 raft_consensus.cc:740] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.367835 20179 consensus_queue.cc:260] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.368327 20177 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.368878 20177 ts_tablet_manager.cc:1403] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.000s
I20260709 13:04:51.370362 20179 ts_tablet_manager.cc:1434] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:04:51.374461 20178 tablet_bootstrap.cc:654] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.376976 19884 raft_consensus.cc:2468] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:51.377094 20212 consensus_queue.cc:1048] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.378576 19924 leader_election.cc:304] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.381450 20212 raft_consensus.cc:2804] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.381564 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05ac482d0e14fc99e404ff234bd53fa" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.381976 20212 raft_consensus.cc:697] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:51.382320 20032 raft_consensus.cc:3060] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.382880 20212 consensus_queue.cc:237] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.371915 20177 raft_consensus.cc:359] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.386276 20177 raft_consensus.cc:385] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.386633 20177 raft_consensus.cc:740] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.387825 20177 consensus_queue.cc:260] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.391577 20032 raft_consensus.cc:2468] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 1.
I20260709 13:04:51.392643 20182 leader_election.cc:290] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.395340 20177 ts_tablet_manager.cc:1434] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 13:04:51.396420 20177 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.396507 20178 tablet_bootstrap.cc:492] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: No bootstrap required, opened a new log
I20260709 13:04:51.397356 20178 ts_tablet_manager.cc:1403] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.006s
I20260709 13:04:51.400861 20178 raft_consensus.cc:359] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.401818 20178 raft_consensus.cc:385] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.402216 20178 raft_consensus.cc:740] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.402907 20178 consensus_queue.cc:260] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.403426 20177 tablet_bootstrap.cc:654] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.405709 20178 ts_tablet_manager.cc:1434] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:51.417665 20211 consensus_queue.cc:1048] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.418778 20177 tablet_bootstrap.cc:492] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.419314 20177 ts_tablet_manager.cc:1403] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.000s
I20260709 13:04:51.419557 19783 catalog_manager.cc:5697] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 1 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.422132 20177 raft_consensus.cc:359] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.422860 20177 raft_consensus.cc:385] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.423086 20177 raft_consensus.cc:740] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.423705 20177 consensus_queue.cc:260] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.425423 20177 ts_tablet_manager.cc:1434] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:51.428679 20194 raft_consensus.cc:493] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.429509 20177 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: Bootstrap starting.
I20260709 13:04:51.429236 20194 raft_consensus.cc:515] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.431353 20194 leader_election.cc:290] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.432866 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.433545 19959 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:51.433818 19883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.434514 19883 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:51.434777 20000 leader_election.cc:304] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:51.435693 20194 raft_consensus.cc:2804] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.436044 20194 raft_consensus.cc:493] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.436372 20194 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.436457 20177 tablet_bootstrap.cc:654] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:51.443884 20177 tablet_bootstrap.cc:492] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: No bootstrap required, opened a new log
I20260709 13:04:51.443670 20194 raft_consensus.cc:515] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.444398 20177 ts_tablet_manager.cc:1403] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.005s
I20260709 13:04:51.446484 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.446802 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.447379 19959 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.447082 20177 raft_consensus.cc:359] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.447853 20177 raft_consensus.cc:385] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:51.448136 20177 raft_consensus.cc:740] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Initialized, Role: FOLLOWER
I20260709 13:04:51.447127 19884 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.448805 20177 consensus_queue.cc:260] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.451004 20177 ts_tablet_manager.cc:1434] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:51.454236 19959 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:51.454782 19884 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:51.454974 20194 leader_election.cc:290] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.455447 20000 leader_election.cc:304] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, 79b099cf2a294ddea68376a11141032f; no voters: 
I20260709 13:04:51.456414 20194 raft_consensus.cc:2804] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.456808 20194 raft_consensus.cc:697] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:51.457535 20194 consensus_queue.cc:237] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.464140 19781 catalog_manager.cc:5697] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.501595 20209 raft_consensus.cc:493] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.502260 20209 raft_consensus.cc:515] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.502877 20194 raft_consensus.cc:493] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.503391 20194 raft_consensus.cc:515] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.504808 20209 leader_election.cc:290] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.505460 20194 leader_election.cc:290] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.505093 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.505415 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" is_pre_election: true
I20260709 13:04:51.506100 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6" is_pre_election: true
I20260709 13:04:51.506433 19959 raft_consensus.cc:2468] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:51.506716 20032 raft_consensus.cc:2468] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 0.
I20260709 13:04:51.507103 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f" is_pre_election: true
I20260709 13:04:51.507965 19851 leader_election.cc:304] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.506958 19884 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:51.508930 20209 raft_consensus.cc:2804] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.509056 19958 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 0.
I20260709 13:04:51.509526 19998 leader_election.cc:304] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.510087 20209 raft_consensus.cc:493] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.510562 20194 raft_consensus.cc:2804] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:51.511019 20194 raft_consensus.cc:493] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.510592 20209 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.511437 20194 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.518724 20209 raft_consensus.cc:515] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.518724 20194 raft_consensus.cc:515] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.520767 20209 leader_election.cc:290] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.521054 20194 leader_election.cc:290] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:51.521763 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.521766 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.522578 19884 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.522634 19958 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.522509 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.522655 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.523614 20032 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.523808 19959 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:51.529894 19884 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:51.529894 19958 raft_consensus.cc:2468] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.531004 20032 raft_consensus.cc:2468] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 1.
I20260709 13:04:51.531265 19998 leader_election.cc:304] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.531266 19959 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 1.
I20260709 13:04:51.532086 19851 leader_election.cc:304] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.534092 20209 raft_consensus.cc:2804] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.534577 20209 raft_consensus.cc:697] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:51.535514 20194 raft_consensus.cc:2804] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:51.535360 20209 consensus_queue.cc:237] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.536087 20194 raft_consensus.cc:697] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:51.537045 20194 consensus_queue.cc:237] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.544338 19781 catalog_manager.cc:5697] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 1 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.545539 19783 catalog_manager.cc:5697] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 1 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.648849 20194 consensus_queue.cc:1048] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.651451 20182 consensus_queue.cc:1048] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.670023 20194 consensus_queue.cc:1048] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.673185 20083 consensus_queue.cc:1048] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.778650 19958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
mode: GRACEFUL
new_leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:39714
I20260709 13:04:51.779273 19958 raft_consensus.cc:606] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Received request to transfer leadership to TS aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:51.784624 19958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "541c795c5782411c8c8165273ea72e97"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
mode: GRACEFUL
new_leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:39714
I20260709 13:04:51.785151 19958 raft_consensus.cc:606] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Received request to transfer leadership to TS aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:51.793197 20033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:51.793843 20033 raft_consensus.cc:606] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS aa705cbdb57641be99f49d5e3c4225e6
I20260709 13:04:51.798662 20209 consensus_queue.cc:1048] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.799789 19883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:55528
I20260709 13:04:51.800442 19883 raft_consensus.cc:606] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.802959 20209 consensus_queue.cc:1048] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.826939 20032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:51.827760 20032 raft_consensus.cc:606] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.836793 20032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
mode: GRACEFUL
new_leader_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:51488
I20260709 13:04:51.837486 20032 raft_consensus.cc:606] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Received request to transfer leadership to TS 79b099cf2a294ddea68376a11141032f
I20260709 13:04:51.838465 20194 consensus_queue.cc:1048] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.844069 19884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
mode: GRACEFUL
new_leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:55528
I20260709 13:04:51.844624 19884 raft_consensus.cc:606] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Received request to transfer leadership to TS 1bc1ec1548ac4dbc933b527c9d4103fb
I20260709 13:04:51.850585 20113 consensus_queue.cc:1048] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.862818 20113 raft_consensus.cc:993] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:51.863306 20113 raft_consensus.cc:1081] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:51.865829 19958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:04:51.866528 19958 raft_consensus.cc:493] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.866886 19958 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.872231 19958 raft_consensus.cc:515] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.875527 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.875540 19958 leader_election.cc:290] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.876168 19884 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.879398 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a9b4b7f33f94c29b99a114301ee76d5" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.880085 20032 raft_consensus.cc:3055] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.880399 20032 raft_consensus.cc:740] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:51.881201 20032 consensus_queue.cc:260] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.882428 19884 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:51.882493 20032 raft_consensus.cc:3060] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.883666 19924 leader_election.cc:304] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:51.884403 20182 raft_consensus.cc:2804] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.884850 20182 raft_consensus.cc:697] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:51.885659 20182 consensus_queue.cc:237] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.890443 20032 raft_consensus.cc:2468] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:51.893699 19782 catalog_manager.cc:5697] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.913646 20188 consensus_queue.cc:1048] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.920094 20238 raft_consensus.cc:993] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:51.920677 20209 raft_consensus.cc:1081] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:51.922430 19959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39696
I20260709 13:04:51.923033 19959 raft_consensus.cc:493] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.923436 19959 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.927657 20182 consensus_queue.cc:1048] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.927052 20238 consensus_queue.cc:1048] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:51.931501 19959 raft_consensus.cc:515] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.935375 19959 leader_election.cc:290] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:51.936735 20034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.939512 19883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357cf49a168243adb1cd4a9935593aa9" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:51.940173 19883 raft_consensus.cc:3055] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.940491 19883 raft_consensus.cc:740] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:51.941423 19883 consensus_queue.cc:260] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.942528 20182 consensus_queue.cc:1048] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.942973 19883 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.948865 20113 consensus_queue.cc:1048] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.951347 20182 raft_consensus.cc:993] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: : Instructing follower aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:51.951946 20182 raft_consensus.cc:1081] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Signalling peer aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:51.956210 19883 raft_consensus.cc:2468] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:51.956411 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "541c795c5782411c8c8165273ea72e97"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:55522
I20260709 13:04:51.957319 19885 raft_consensus.cc:493] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.957602 19924 leader_election.cc:304] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 1bc1ec1548ac4dbc933b527c9d4103fb
I20260709 13:04:51.957852 19885 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.959733 20182 raft_consensus.cc:2804] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.960680 20182 raft_consensus.cc:697] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:51.961958 20182 consensus_queue.cc:237] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.967914 19885 raft_consensus.cc:515] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.972345 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:51.973205 19959 raft_consensus.cc:3055] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.973834 19959 raft_consensus.cc:740] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:51.974921 19959 consensus_queue.cc:260] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:51.976167 19959 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.976275 20194 consensus_queue.cc:1048] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.978468 20033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "541c795c5782411c8c8165273ea72e97" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:51.979223 20033 raft_consensus.cc:3060] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.984483 20209 consensus_queue.cc:1048] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.997606 19885 leader_election.cc:290] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:52.007547 20033 raft_consensus.cc:2468] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.008973 19850 leader_election.cc:304] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:52.010017 20209 raft_consensus.cc:2804] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:52.010756 20209 raft_consensus.cc:697] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:52.011649 20209 consensus_queue.cc:237] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.005988 19959 raft_consensus.cc:2468] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.018199 20247 raft_consensus.cc:993] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:52.018630 20247 raft_consensus.cc:1081] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer 79b099cf2a294ddea68376a11141032f to start an election
I20260709 13:04:52.021078 19958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:04:52.021728 19958 raft_consensus.cc:493] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:52.022106 19958 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.023240 20238 raft_consensus.cc:993] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: : Instructing follower 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
I20260709 13:04:52.023792 20238 raft_consensus.cc:1081] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Signalling peer 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
I20260709 13:04:52.023339 19782 catalog_manager.cc:5697] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.028581 20033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:51470
I20260709 13:04:52.029928 19958 raft_consensus.cc:515] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.030588 20238 consensus_queue.cc:1048] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:52.031847 20033 raft_consensus.cc:493] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:52.032223 20033 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.032896 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:52.033567 19884 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.039286 20033 raft_consensus.cc:515] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.040434 19884 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:52.041663 19924 leader_election.cc:304] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:52.043318 19958 leader_election.cc:290] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:52.044786 19884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:52.045433 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b7f2c14823c4d7c9350fe2546e4141d" candidate_uuid: "79b099cf2a294ddea68376a11141032f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:52.046118 20032 raft_consensus.cc:3055] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:52.046581 20032 raft_consensus.cc:740] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:52.045439 19884 raft_consensus.cc:3055] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:52.048348 19884 raft_consensus.cc:740] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:52.049963 19884 consensus_queue.cc:260] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.050817 19884 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.047544 20032 consensus_queue.cc:260] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.052405 20032 raft_consensus.cc:3060] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.053047 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbafbdd4378a47cd8fdbb28076f91c85" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:52.057235 20182 raft_consensus.cc:2804] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:52.058056 20182 raft_consensus.cc:697] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Leader. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:52.058923 20182 consensus_queue.cc:237] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.057379 20033 leader_election.cc:290] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:52.068202 19884 raft_consensus.cc:2468] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 2.
I20260709 13:04:52.069556 19998 leader_election.cc:304] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 79b099cf2a294ddea68376a11141032f
I20260709 13:04:52.069799 19781 catalog_manager.cc:5697] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 1 to 2, leader changed from 79b099cf2a294ddea68376a11141032f (127.7.117.130) to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.070847 20247 raft_consensus.cc:2804] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:52.071336 20247 raft_consensus.cc:697] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:52.072105 20247 consensus_queue.cc:237] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.072688 20032 raft_consensus.cc:2468] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79b099cf2a294ddea68376a11141032f in term 2.
I20260709 13:04:52.081143 19782 catalog_manager.cc:5697] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to 79b099cf2a294ddea68376a11141032f (127.7.117.130). New cstate: current_term: 2 leader_uuid: "79b099cf2a294ddea68376a11141032f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.089844 19780 catalog_manager.cc:5697] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 1 to 2, leader changed from aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129) to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:52.229919 20212 raft_consensus.cc:993] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: : Instructing follower aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:52.230346 20182 raft_consensus.cc:1081] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Signalling peer aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:52.231947 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:55522
I20260709 13:04:52.232436 19884 raft_consensus.cc:493] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:52.232725 19884 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.237241 19884 raft_consensus.cc:515] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.238961 19884 leader_election.cc:290] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:52.239755 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:52.239969 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849f14f2ac14be59c3f5ce389ee5504" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:52.240404 19958 raft_consensus.cc:3055] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:52.240583 20032 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.240732 19958 raft_consensus.cc:740] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:52.241575 19958 consensus_queue.cc:260] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.242482 19958 raft_consensus.cc:3060] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.246908 20032 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.248086 20194 raft_consensus.cc:993] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: : Instructing follower aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:52.248685 20194 raft_consensus.cc:1081] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Signalling peer aa705cbdb57641be99f49d5e3c4225e6 to start an election
I20260709 13:04:52.249117 19850 leader_election.cc:304] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:52.249421 19958 raft_consensus.cc:2468] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.250051 20238 raft_consensus.cc:2804] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:52.250473 19884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
 from {username='slave'} at 127.0.0.1:55512
I20260709 13:04:52.251075 19884 raft_consensus.cc:493] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:52.250620 20238 raft_consensus.cc:697] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:52.251390 19884 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.251966 20238 consensus_queue.cc:237] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.256696 19884 raft_consensus.cc:515] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.258364 19884 leader_election.cc:290] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Requested vote from peers 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275), 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131:35871)
I20260709 13:04:52.259168 19958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:52.258554 19782 catalog_manager.cc:5697] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 1 to 2, leader changed from 79b099cf2a294ddea68376a11141032f (127.7.117.130) to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.259723 19958 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.259768 20032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ba5601b329454a92b5fc4e4276714c" candidate_uuid: "aa705cbdb57641be99f49d5e3c4225e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
I20260709 13:04:52.260438 20032 raft_consensus.cc:3055] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:52.260725 20032 raft_consensus.cc:740] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:52.261524 20032 consensus_queue.cc:260] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.262822 20032 raft_consensus.cc:3060] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.267990 19958 raft_consensus.cc:2468] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.268250 20032 raft_consensus.cc:2468] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa705cbdb57641be99f49d5e3c4225e6 in term 2.
I20260709 13:04:52.269178 19851 leader_election.cc:304] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79b099cf2a294ddea68376a11141032f, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:52.270498 20238 raft_consensus.cc:2804] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:52.271215 20238 raft_consensus.cc:697] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Becoming Leader. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:52.272423 20238 consensus_queue.cc:237] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:52.279553 19782 catalog_manager.cc:5697] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 reported cstate change: term changed from 1 to 2, leader changed from 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131) to aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129). New cstate: current_term: 2 leader_uuid: "aa705cbdb57641be99f49d5e3c4225e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.372889 19884 raft_consensus.cc:1275] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.374313 20212 consensus_queue.cc:1048] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.382522 20032 raft_consensus.cc:3060] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.386720 19885 raft_consensus.cc:1275] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.389662 20032 raft_consensus.cc:1275] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.390579 20211 consensus_queue.cc:1048] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.391222 20182 consensus_queue.cc:1048] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.399997 20032 raft_consensus.cc:1275] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.402535 20182 consensus_queue.cc:1048] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.497180 19958 raft_consensus.cc:1275] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.498462 20238 consensus_queue.cc:1048] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.508069 20032 raft_consensus.cc:1275] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.509543 20252 consensus_queue.cc:1048] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.565189 19885 raft_consensus.cc:1275] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.566555 20083 consensus_queue.cc:1048] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.570387 19958 raft_consensus.cc:3060] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:52.576504 20032 raft_consensus.cc:1275] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer 79b099cf2a294ddea68376a11141032f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.578351 19958 raft_consensus.cc:1275] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.578989 20212 consensus_queue.cc:1048] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.579844 20194 consensus_queue.cc:1048] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.599099 19885 raft_consensus.cc:1275] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.600835 20247 consensus_queue.cc:1048] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.725410 20032 raft_consensus.cc:1275] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.726545 20188 consensus_queue.cc:1048] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.736518 19958 raft_consensus.cc:1275] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.737768 20238 consensus_queue.cc:1048] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.804454 19958 raft_consensus.cc:1275] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.805578 20238 consensus_queue.cc:1048] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.815894 20032 raft_consensus.cc:1275] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Refusing update from remote peer aa705cbdb57641be99f49d5e3c4225e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.817533 20238 consensus_queue.cc:1048] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:54.847910 19782 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36808:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:04:54.866243 19782 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36808:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:04:54.980412  7638 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:55.024049 20303 tablet.cc:1724] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.024943 20297 tablet.cc:1724] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.042598 20298 tablet.cc:1724] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.050894 20302 tablet.cc:1724] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.059747 20301 tablet.cc:1724] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.129551 19883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b"
dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
mode: GRACEFUL
new_leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:60424
I20260709 13:04:55.130264 19883 raft_consensus.cc:606] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Received request to transfer leadership to TS 1bc1ec1548ac4dbc933b527c9d4103fb
I20260709 13:04:55.184214 20303 tablet.cc:1724] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.210855 20298 tablet.cc:1724] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.240895 20306 log.cc:927] Time spent T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:04:55.242229 20313 log.cc:927] Time spent T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:04:55.245554 20309 log.cc:927] Time spent T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.004s
I20260709 13:04:55.247270 20281 raft_consensus.cc:993] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: : Instructing follower 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
W20260709 13:04:55.247433 20310 log.cc:927] Time spent T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:04:55.247704 20281 raft_consensus.cc:1081] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Signalling peer 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
W20260709 13:04:55.250363 20311 log.cc:927] Time spent T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:04:55.250684 20032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:51470
I20260709 13:04:55.251268 20032 raft_consensus.cc:493] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.251646 20032 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.249632 20318 tablet.cc:1724] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.275691 20317 tablet.cc:1724] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.277983 20301 tablet.cc:1724] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.276211 20302 tablet.cc:1724] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.279654 19998 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.129:34441, user_credentials={real_user=slave}} blocked reactor thread for 55984.5us
I20260709 13:04:55.304214 20032 raft_consensus.cc:515] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
W20260709 13:04:55.276854 20315 log.cc:927] Time spent T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:55.314494 20297 tablet.cc:1724] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.328433 20330 tablet.cc:1724] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.325433 20032 leader_election.cc:290] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 2 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:55.338297 19882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:55.339242 19882 raft_consensus.cc:3055] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:55.339654 19882 raft_consensus.cc:740] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa705cbdb57641be99f49d5e3c4225e6, State: Running, Role: LEADER
I20260709 13:04:55.341581 19882 consensus_queue.cc:260] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:55.343442 19882 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.352058 20314 tablet.cc:1724] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.335570 20303 tablet.cc:1724] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.360844 20318 tablet.cc:1724] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.360900 20298 tablet.cc:1724] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.366874 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45de4865bd314c65b42a7121fdd2720b" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:55.367784 19959 raft_consensus.cc:3060] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.273238 20327 tablet.cc:1724] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.375443 20302 tablet.cc:1724] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.380664 20317 tablet.cc:1724] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.274329 20328 tablet.cc:1724] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.399871 20301 tablet.cc:1724] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.414585 20318 tablet.cc:1724] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.393103 20331 tablet.cc:1724] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.405503 19925 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.117.131:35871, user_credentials={real_user=slave}} blocked reactor thread for 72499.8us
I20260709 13:04:55.412855 19960 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "713e842bd00941b4b931176299bc5ada"
dest_uuid: "79b099cf2a294ddea68376a11141032f"
mode: GRACEFUL
new_leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:42298
I20260709 13:04:55.412992 20330 tablet.cc:1724] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.428964 19960 raft_consensus.cc:606] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Received request to transfer leadership to TS 1bc1ec1548ac4dbc933b527c9d4103fb
I20260709 13:04:55.402813 19882 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 2.
I20260709 13:04:55.430835 19998 leader_election.cc:304] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:55.431887 20260 raft_consensus.cc:2804] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:55.432426 20260 raft_consensus.cc:697] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:55.431052 19959 raft_consensus.cc:2468] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 2.
I20260709 13:04:55.433434 20260 consensus_queue.cc:237] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:55.425276 20301 tablet.cc:1724] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.439196  7638 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:55.439859  7638 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:55.443481 20327 tablet.cc:1724] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.475189 19781 catalog_manager.cc:6242] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: soft_deleted_table [id=a1806af7a75f4544a0f80343a0f82c3b] alter complete (version 1)
I20260709 13:04:55.486086 19780 catalog_manager.cc:5697] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 1 to 2, leader changed from aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129) to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 2 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.636884 20332 raft_consensus.cc:993] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: : Instructing follower 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
I20260709 13:04:55.637406 20083 raft_consensus.cc:1081] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Signalling peer 1bc1ec1548ac4dbc933b527c9d4103fb to start an election
I20260709 13:04:55.638926 20029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "713e842bd00941b4b931176299bc5ada"
dest_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb"
 from {username='slave'} at 127.0.0.1:51474
I20260709 13:04:55.639422 20029 raft_consensus.cc:493] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.639740 20029 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.646212 20029 raft_consensus.cc:515] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:55.648334 20029 leader_election.cc:290] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 3 election: Requested vote from peers aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441), 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275)
I20260709 13:04:55.649034 19882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "aa705cbdb57641be99f49d5e3c4225e6"
I20260709 13:04:55.649766 19882 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.649909 19959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713e842bd00941b4b931176299bc5ada" candidate_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79b099cf2a294ddea68376a11141032f"
I20260709 13:04:55.650571 19959 raft_consensus.cc:3055] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.650831 19959 raft_consensus.cc:740] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 79b099cf2a294ddea68376a11141032f, State: Running, Role: LEADER
I20260709 13:04:55.651615 19959 consensus_queue.cc:260] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:55.652791 19959 raft_consensus.cc:3060] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.655893 19882 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 3.
I20260709 13:04:55.657063 19998 leader_election.cc:304] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc1ec1548ac4dbc933b527c9d4103fb, aa705cbdb57641be99f49d5e3c4225e6; no voters: 
I20260709 13:04:55.657963 20260 raft_consensus.cc:2804] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.658413 19959 raft_consensus.cc:2468] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc1ec1548ac4dbc933b527c9d4103fb in term 3.
I20260709 13:04:55.658424 20260 raft_consensus.cc:697] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 3 LEADER]: Becoming Leader. State: Replica: 1bc1ec1548ac4dbc933b527c9d4103fb, State: Running, Role: LEADER
I20260709 13:04:55.660542 20260 consensus_queue.cc:237] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } }
I20260709 13:04:55.671737 19780 catalog_manager.cc:5697] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb reported cstate change: term changed from 2 to 3, leader changed from 79b099cf2a294ddea68376a11141032f (127.7.117.130) to 1bc1ec1548ac4dbc933b527c9d4103fb (127.7.117.131). New cstate: current_term: 3 leader_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc1ec1548ac4dbc933b527c9d4103fb" member_type: VOTER last_known_addr { host: "127.7.117.131" port: 35871 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.980640 19885 raft_consensus.cc:1275] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:55.982156 20260 consensus_queue.cc:1048] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.998728 19959 raft_consensus.cc:1275] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:56.000396 20260 consensus_queue.cc:1048] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:56.144800 19885 raft_consensus.cc:1275] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 3 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:56.146010 20326 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa705cbdb57641be99f49d5e3c4225e6" member_type: VOTER last_known_addr { host: "127.7.117.129" port: 34441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.154441 19959 raft_consensus.cc:1275] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 3 FOLLOWER]: Refusing update from remote peer 1bc1ec1548ac4dbc933b527c9d4103fb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:56.156164 20326 consensus_queue.cc:1048] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b099cf2a294ddea68376a11141032f" member_type: VOTER last_known_addr { host: "127.7.117.130" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.497519  7638 tablet_server.cc:179] TabletServer@127.7.117.129:0 shutting down...
W20260709 13:04:56.522893 19924 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
I20260709 13:04:56.525259  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:56.526314  7638 tablet_replica.cc:333] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
W20260709 13:04:56.526836 19924 consensus_peers.cc:597] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f -> Peer aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441): Couldn't send request to peer aa705cbdb57641be99f49d5e3c4225e6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.527169  7638 raft_consensus.cc:2243] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.527680  7638 raft_consensus.cc:2272] T 3b7f2c14823c4d7c9350fe2546e4141d P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.529528  7638 tablet_replica.cc:333] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.530170  7638 raft_consensus.cc:2243] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.530683  7638 raft_consensus.cc:2272] T cbafbdd4378a47cd8fdbb28076f91c85 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.533145  7638 tablet_replica.cc:333] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.533878  7638 raft_consensus.cc:2243] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.534396  7638 raft_consensus.cc:2272] T a05ac482d0e14fc99e404ff234bd53fa P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.536170  7638 tablet_replica.cc:333] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.536679  7638 raft_consensus.cc:2243] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.537048  7638 raft_consensus.cc:2272] T 3a9b4b7f33f94c29b99a114301ee76d5 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.538820  7638 tablet_replica.cc:333] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.539314  7638 raft_consensus.cc:2243] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.540004  7638 raft_consensus.cc:2272] T e5ba5601b329454a92b5fc4e4276714c P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.541792  7638 tablet_replica.cc:333] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.542339  7638 raft_consensus.cc:2243] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.543062  7638 raft_consensus.cc:2272] T 541c795c5782411c8c8165273ea72e97 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.544819  7638 tablet_replica.cc:333] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.545429  7638 raft_consensus.cc:2243] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.545887  7638 raft_consensus.cc:2272] T 153d1a15c28f47b8b822a41ab43d1adc P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.547827  7638 tablet_replica.cc:333] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.548350  7638 raft_consensus.cc:2243] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.548784  7638 raft_consensus.cc:2272] T 28c8e89b7ec540cb9eda24fd98ef7acc P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.551200  7638 tablet_replica.cc:333] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.551960  7638 raft_consensus.cc:2243] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.552538  7638 raft_consensus.cc:2272] T 713e842bd00941b4b931176299bc5ada P aa705cbdb57641be99f49d5e3c4225e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.555219  7638 tablet_replica.cc:333] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.555862  7638 raft_consensus.cc:2243] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.556597  7638 raft_consensus.cc:2272] T 8849f14f2ac14be59c3f5ce389ee5504 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.558377  7638 tablet_replica.cc:333] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.558904  7638 raft_consensus.cc:2243] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.559327  7638 raft_consensus.cc:2272] T 357cf49a168243adb1cd4a9935593aa9 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.561044  7638 tablet_replica.cc:333] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.561623  7638 raft_consensus.cc:2243] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.562062  7638 raft_consensus.cc:2272] T 0b0cf2c1fe954feba34a9cb23532cbca P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.563861  7638 tablet_replica.cc:333] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.564389  7638 raft_consensus.cc:2243] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.564790  7638 raft_consensus.cc:2272] T 45de4865bd314c65b42a7121fdd2720b P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.567267  7638 tablet_replica.cc:333] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.567824  7638 raft_consensus.cc:2243] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.568560  7638 raft_consensus.cc:2272] T 0d98942d5e644d14965245d02d872469 P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.570372  7638 tablet_replica.cc:333] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.570950  7638 raft_consensus.cc:2243] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.571380  7638 raft_consensus.cc:2272] T dfbbbe561e014873b8e6fe086f3a8b31 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.573168  7638 tablet_replica.cc:333] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.573668  7638 raft_consensus.cc:2243] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.574440  7638 raft_consensus.cc:2272] T 5041e1d57a3c4ec88074674cabbdf7ef P aa705cbdb57641be99f49d5e3c4225e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.576443  7638 tablet_replica.cc:333] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.577109  7638 raft_consensus.cc:2243] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.578007  7638 raft_consensus.cc:2272] T d601750e621f4e9a83d1153e5b946a7e P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.580502  7638 tablet_replica.cc:333] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6: stopping tablet replica
I20260709 13:04:56.580979  7638 raft_consensus.cc:2243] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.581769  7638 raft_consensus.cc:2272] T 2d08efba25a74cb69708233eef807713 P aa705cbdb57641be99f49d5e3c4225e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.623368  7638 tablet_server.cc:196] TabletServer@127.7.117.129:0 shutdown complete.
I20260709 13:04:56.655526  7638 tablet_server.cc:179] TabletServer@127.7.117.130:0 shutting down...
W20260709 13:04:56.664680 19924 consensus_peers.cc:597] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f -> Peer aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441): Couldn't send request to peer aa705cbdb57641be99f49d5e3c4225e6. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:34441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.672782 19998 consensus_peers.cc:597] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb -> Peer aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441): Couldn't send request to peer aa705cbdb57641be99f49d5e3c4225e6. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:34441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.682124 19924 consensus_peers.cc:597] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f -> Peer aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441): Couldn't send request to peer aa705cbdb57641be99f49d5e3c4225e6. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:34441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:56.686975 19998 consensus_peers.cc:597] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb -> Peer aa705cbdb57641be99f49d5e3c4225e6 (127.7.117.129:34441): Couldn't send request to peer aa705cbdb57641be99f49d5e3c4225e6. Status: Network error: Client connection negotiation failed: client connection to 127.7.117.129:34441: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.692806  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:56.693589  7638 tablet_replica.cc:333] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.694231  7638 raft_consensus.cc:2243] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.694800  7638 raft_consensus.cc:2272] T cbafbdd4378a47cd8fdbb28076f91c85 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.696777  7638 tablet_replica.cc:333] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.697639  7638 raft_consensus.cc:2243] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.698467  7638 raft_consensus.cc:2272] T 3b7f2c14823c4d7c9350fe2546e4141d P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.700224  7638 tablet_replica.cc:333] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.700867  7638 raft_consensus.cc:2243] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.701695  7638 raft_consensus.cc:2272] T a05ac482d0e14fc99e404ff234bd53fa P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.703557  7638 tablet_replica.cc:333] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.704107  7638 raft_consensus.cc:2243] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.704834  7638 raft_consensus.cc:2272] T 3a9b4b7f33f94c29b99a114301ee76d5 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.706688  7638 tablet_replica.cc:333] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.707249  7638 raft_consensus.cc:2243] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.707720  7638 raft_consensus.cc:2272] T e5ba5601b329454a92b5fc4e4276714c P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.709537  7638 tablet_replica.cc:333] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.710178  7638 raft_consensus.cc:2243] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.710610  7638 raft_consensus.cc:2272] T 8849f14f2ac14be59c3f5ce389ee5504 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.712320  7638 tablet_replica.cc:333] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.712859  7638 raft_consensus.cc:2243] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.713279  7638 raft_consensus.cc:2272] T 541c795c5782411c8c8165273ea72e97 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.715090  7638 tablet_replica.cc:333] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.715641  7638 raft_consensus.cc:2243] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.716359  7638 raft_consensus.cc:2272] T 357cf49a168243adb1cd4a9935593aa9 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.718181  7638 tablet_replica.cc:333] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.718742  7638 raft_consensus.cc:2243] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.719535  7638 raft_consensus.cc:2272] T 0b0cf2c1fe954feba34a9cb23532cbca P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.721529  7638 tablet_replica.cc:333] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.722075  7638 raft_consensus.cc:2243] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.722829  7638 raft_consensus.cc:2272] T 28c8e89b7ec540cb9eda24fd98ef7acc P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.724828  7638 tablet_replica.cc:333] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.725481  7638 raft_consensus.cc:2243] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.725841  7638 raft_consensus.cc:2272] T 713e842bd00941b4b931176299bc5ada P 79b099cf2a294ddea68376a11141032f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.728838  7638 tablet_replica.cc:333] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.729436  7638 raft_consensus.cc:2243] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.729862  7638 raft_consensus.cc:2272] T 153d1a15c28f47b8b822a41ab43d1adc P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.731871  7638 tablet_replica.cc:333] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.732390  7638 raft_consensus.cc:2243] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.732810  7638 raft_consensus.cc:2272] T 5041e1d57a3c4ec88074674cabbdf7ef P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.734733  7638 tablet_replica.cc:333] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.735241  7638 raft_consensus.cc:2243] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.735646  7638 raft_consensus.cc:2272] T d601750e621f4e9a83d1153e5b946a7e P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.737656  7638 tablet_replica.cc:333] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.738155  7638 raft_consensus.cc:2243] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.738554  7638 raft_consensus.cc:2272] T 45de4865bd314c65b42a7121fdd2720b P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.740695  7638 tablet_replica.cc:333] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.741227  7638 raft_consensus.cc:2243] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.742033  7638 raft_consensus.cc:2272] T dfbbbe561e014873b8e6fe086f3a8b31 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.743881  7638 tablet_replica.cc:333] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.744349  7638 raft_consensus.cc:2243] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.744767  7638 raft_consensus.cc:2272] T 0d98942d5e644d14965245d02d872469 P 79b099cf2a294ddea68376a11141032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.746670  7638 tablet_replica.cc:333] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f: stopping tablet replica
I20260709 13:04:56.747136  7638 raft_consensus.cc:2243] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.747542  7638 raft_consensus.cc:2272] T 2d08efba25a74cb69708233eef807713 P 79b099cf2a294ddea68376a11141032f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:56.753644 20000 consensus_peers.cc:597] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb -> Peer 79b099cf2a294ddea68376a11141032f (127.7.117.130:39275): Couldn't send request to peer 79b099cf2a294ddea68376a11141032f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.787885  7638 tablet_server.cc:196] TabletServer@127.7.117.130:0 shutdown complete.
I20260709 13:04:56.818619  7638 tablet_server.cc:179] TabletServer@127.7.117.131:0 shutting down...
I20260709 13:04:56.849720  7638 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:56.850404  7638 tablet_replica.cc:333] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.851037  7638 raft_consensus.cc:2243] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.851858  7638 raft_consensus.cc:2272] T cbafbdd4378a47cd8fdbb28076f91c85 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.853829  7638 tablet_replica.cc:333] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.854382  7638 raft_consensus.cc:2243] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.854809  7638 raft_consensus.cc:2272] T a05ac482d0e14fc99e404ff234bd53fa P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.856556  7638 tablet_replica.cc:333] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.857097  7638 raft_consensus.cc:2243] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.857546  7638 raft_consensus.cc:2272] T 3b7f2c14823c4d7c9350fe2546e4141d P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.859251  7638 tablet_replica.cc:333] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.859776  7638 raft_consensus.cc:2243] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.860210  7638 raft_consensus.cc:2272] T 8849f14f2ac14be59c3f5ce389ee5504 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.861976  7638 tablet_replica.cc:333] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.862490  7638 raft_consensus.cc:2243] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.862897  7638 raft_consensus.cc:2272] T 541c795c5782411c8c8165273ea72e97 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.864524  7638 tablet_replica.cc:333] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.865067  7638 raft_consensus.cc:2243] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.865531  7638 raft_consensus.cc:2272] T 28c8e89b7ec540cb9eda24fd98ef7acc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.867467  7638 tablet_replica.cc:333] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.868011  7638 raft_consensus.cc:2243] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.868777  7638 raft_consensus.cc:2272] T 713e842bd00941b4b931176299bc5ada P 1bc1ec1548ac4dbc933b527c9d4103fb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.871060  7638 tablet_replica.cc:333] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.871625  7638 raft_consensus.cc:2243] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.872063  7638 raft_consensus.cc:2272] T 357cf49a168243adb1cd4a9935593aa9 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.873811  7638 tablet_replica.cc:333] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.874425  7638 raft_consensus.cc:2243] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.874817  7638 raft_consensus.cc:2272] T 0b0cf2c1fe954feba34a9cb23532cbca P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.876641  7638 tablet_replica.cc:333] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.877146  7638 raft_consensus.cc:2243] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.877971  7638 raft_consensus.cc:2272] T 153d1a15c28f47b8b822a41ab43d1adc P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.879853  7638 tablet_replica.cc:333] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.880368  7638 raft_consensus.cc:2243] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.880785  7638 raft_consensus.cc:2272] T 3a9b4b7f33f94c29b99a114301ee76d5 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.882506  7638 tablet_replica.cc:333] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.883025  7638 raft_consensus.cc:2243] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.883442  7638 raft_consensus.cc:2272] T e5ba5601b329454a92b5fc4e4276714c P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.885092  7638 tablet_replica.cc:333] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.885627  7638 raft_consensus.cc:2243] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.886031  7638 raft_consensus.cc:2272] T dfbbbe561e014873b8e6fe086f3a8b31 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.887687  7638 tablet_replica.cc:333] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.888171  7638 raft_consensus.cc:2243] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.888624  7638 raft_consensus.cc:2272] T 0d98942d5e644d14965245d02d872469 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.890478  7638 tablet_replica.cc:333] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.891021  7638 raft_consensus.cc:2243] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.891444  7638 raft_consensus.cc:2272] T 2d08efba25a74cb69708233eef807713 P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.893435  7638 tablet_replica.cc:333] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.893939  7638 raft_consensus.cc:2243] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.894440  7638 raft_consensus.cc:2272] T d601750e621f4e9a83d1153e5b946a7e P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.896442  7638 tablet_replica.cc:333] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.896947  7638 raft_consensus.cc:2243] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.897410  7638 raft_consensus.cc:2272] T 5041e1d57a3c4ec88074674cabbdf7ef P 1bc1ec1548ac4dbc933b527c9d4103fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.899374  7638 tablet_replica.cc:333] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb: stopping tablet replica
I20260709 13:04:56.899830  7638 raft_consensus.cc:2243] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.900559  7638 raft_consensus.cc:2272] T 45de4865bd314c65b42a7121fdd2720b P 1bc1ec1548ac4dbc933b527c9d4103fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.940222  7638 tablet_server.cc:196] TabletServer@127.7.117.131:0 shutdown complete.
I20260709 13:04:56.968926  7638 master.cc:562] Master@127.7.117.190:42699 shutting down...
I20260709 13:04:56.991258  7638 raft_consensus.cc:2243] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:56.991852  7638 raft_consensus.cc:2272] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.992235  7638 tablet_replica.cc:333] T 00000000000000000000000000000000 P de635c90a3884ef79a0dc62fe61f6008: stopping tablet replica
I20260709 13:04:57.013185  7638 master.cc:584] Master@127.7.117.190:42699 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11395 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24477 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (361024 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:04:57.068504  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 409 messages since previous log ~14 seconds ago
I20260709 13:04:57.068758  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 409 messages since previous log ~14 seconds ago
I20260709 13:04:57.068962  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 406 messages since previous log ~14 seconds ago
I20260709 13:04:57.069175  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~36 seconds ago
I20260709 13:04:57.069658  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~36 seconds ago
I20260709 13:04:57.069897  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~131 seconds ago
I20260709 13:04:57.070164  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 5 messages since previous log ~0 seconds ago
I20260709 13:04:57.070493  7638 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~10 seconds ago
